ansible

Ansible pushes IT Infra / Config from a control node using SSH
Commands are parallelised between hosts
Ordering can be important, watch out for deadlocks
Ansible is written in Python

=====
Inventory File

- Lives at /etc/ansible/hosts by default
- Can be flat file
- Can be script
- It's best practice to include the control host
- There is always an "all" group

//Lots of different ways to select hosts as exemplified here
ansible --list-hosts <groupOrHostName> //Query hosts from CLI - can use blobbing
ansible --list-hosts "<fragment>*" //Can use globbing
ansible --list-hosts <groupOrHostName>,<groupOrHostName>... //Can use comma sep list (or colon sep on old versions)
ansible --list-hosts <group>[n] //Can select one from a group by position (0 indexed)
ansible --list-hosts \!<anythingAbove> //Everything but selection (escape in bash etc)
//Use -i <filename> flag to use anything but the default inventory file (or that configured in local ansible.cfg - see below)

Simple Format:
[<groupName>]
<hostname> 
...
[<groupName>]
<hostname> <ansibleVar>=<value> //With a setting
...	

YAML Format (complex example to show all the things):
THIS IS UNTESTED (put is valid YAML)
---
all:
  children:
    tiera:
      hosts:
        52.39.214.106:
        34.222.13.218:
    tierb:
      hosts:
        54.71.85.149:
        34.220.179.211:
    control:
      hosts:
        localhost:
        vars:
          ansible_connection: local
  vars:
    ansible_ssh_private_key_file: /Users/ibodding/.ssh/generic-vm.pem
    ansible_user: centos
    
Ansible Vars:
ansible_connection=local //Don't use SSH for this host

=====
Playbooks

When running a playbook, ansible with gather some facts from each host at the start of the process

Can be viewed as being 4 pillats which constitute most of what needs to be done with Ansible:
1. Packages / Installs
2. Service Handler (systemd...)
3. System Configuration (Dirs, Users...)
4. Application Configuration


ansible-playbook <filename> //Run a playbook
ansible-playbook --syntax-check <fileName> //Check YAML syntax
ansible-playbook --check <fileName> //Dry-Run (limited module support here though)
//Add -v / -vv / -vvv / -vvvv for increasingly verbose output

Example (YAML):
---
  - hosts: <hosts>
    [become: true] //Use sudo for all tasks
    tasks:
      - <module>: <args>
      - name: <name>
        <module>: <args> //Same but with a name / multiple props
        [become: true] //Use sudo for this task
        [with_items]
      ...

EXAMPLES MAKE a 3 tier app (from udemy course)
Example - Using apt to install Nginx (YAML):
//With escalation and service check
---
  - hosts: loadbalancer
    become: true
    tasks:
      - name: install tools
        apt: name={{item}} state=present update_cache=yes
        with_items:
          - python-httplib2
      - name: install nginx
        apt: name=nginx state=present update_cache=yes //Could also use state=latest to upgrade every time ran, or state=absent to remove, update_cache runs apt update first
      - name: ensure nginx started
        service: name-nginx state=started enabled=yes //Check if the service is started, restarts otherwise (can use state=restarted or stopped), enabled checks if on startup
      - name: configure nginx site
        template: src=templates/nginx.conf.j2 dest=/etc/nginx/sites-available/demo mode=0644 //i.e. the example in template section below
        notify: restart nginx        
      - name: de-activate default nginx site
        file: /etc/nginx/sites-enabled/default state=absent
        notify: restart nginx        
      - name: activate demo nginx site
        file: src=/etc/nginx/sites-available/demo dest=/etc/nginx/sites-enabled/demo state=link        
        notify: restart nginx        
    handlers: //Need restart to enable mod_wsgi (but don't want to restart if already in place) - only triggered if notify hit above
      - name: restart nginx
        service: name-nginx state=started enabled=yes                
Example - Using apt to install mysql-server (YAML):
---
  - hosts: database
    tasks:
      - name: install tools
        apt: name={{item}} state=present update_cache=yes
        with_items:
          - python-mysqldb
      - name: install mysql
        apt: "name=mysql-server state=present update_cache=yes" //Could also use state=latest to upgrade every time ran, or state=absent to remove, update_cache runs apt update first
      - name: ensure mysel started
        service: name=mysql-server state=started enabled=yes
      - name ensure mysqle listening on all ports
        lineinfile: dest=/etc/mysql/my.cnf regexp=^bind_address line="bind-address=0.0.0.0"
        notify: restart mysql
      - name: create demo database
        mysql_db: name=demo state=present
      - name: crate demo user
        name=demo password=demo priv=demo.*:ALL host='%' state=present
    handlers:
      - name: restart mysql
        service: name=mysql state=restarted
        
Example - Using apt to install apache2 with dependencies (YAML):
//Using with_items to iterate over a set of with_items - Jinga templating
//Using a handler for apache2 restarts
//Using copy for web application files + virtual host config
//Using pip for web app dependencies
//Using file to touch files on server (for sites-available etc) / symlink
---
  - hosts: webserver
    tasks:
      - name: install apache2
        apt: "name={{item}} state=present update_cache=yes" //Could also use state=latest to upgrade every time ran, or state=absent to remove, update_cache runs apt update first
        with_items:
          - apache2
          - libapache2-mod-wsgi
          - python-pip
          - python-virtualenv
          - python-mysqldb
      - name: ensure nginx started
        service: name-nginx state=started enabled=yes //Check if the service is started, restarts otherwise (can use state=restarted or stopped), enabled checks if on startup          
      - name: ensure mod_wsgi enabled
        apache2_module: state=present name=wsgi
        notify: restart apache2
      - name: copy demo app source
        copy: src=demo/app/ dest=/var/www/demo mode=0755 //trailing slash means directory and contents (not just contents)
        notify: restart apache2
      - name: copy apache virtual host config
        copy: src=demo/demo.conf dest=/etc/apache2/sites-available mode=0755
        notify: restart apache2
      - name: setup python virtualenv //Creates a python virtual env for execution
        pip: requirements=/var/www/demo/requirements.txt virtualenv=/var/www/demo/.venv
        notify: restart apache2
      - name: de-activate default apache site
        file: /etc/apache2/sites-enabled/000-default.conf state=absent
        notify: restart apache2
      - name: activate demo apache site
        file: src=/etc/apache/sites-available/demo.cnf dest=/etc/apache2/sites-enabled/demo.cnf state=link
        notify: restart apache2        
    handlers: //Need restart to enable mod_wsgi (but don't want to restart if already in place) - only triggered if notify hit above
      - name: restart apache2
        service: name-apache2 state=started enabled=yes

Example - status checking for all the things above....
---
  - hosts: loadbalancer
    become: true
    tasks:
      - name: verify nginx status
        command: service nginx status
      - name: verify nginx is listening on 80
        wait_for: port=80 timeout=1 //default is 300 seconds, use state=drained to check for no connections, state-stopped for no listening, state=started (default)
  - hosts: webserver
    become: true
    tasks:
      - name: verify apache2 status
        command: service apache2 status
      - name: verify apache2 is listening on 3306
        wait_for: port=3306 timeout=1 //default is 300 seconds        
  - hosts: database
    become: true
    tasks:
      - name: verify mysql status
        command: service mysql status                
      - name: verify mysql is listening on 80
        wait_for: port=80 timeout=1 //default is 300 seconds        
=====
playbook templates - uses jinger2 templating
-----
Examples:
upstream demo {
{% for server in hroups.webserver %}
  server {{ server }};
{% endfor %}
}
server {
  listen 80;
  
  location / {
    proxy_pass http://demo
  }
}
-----
=====
Parent Playbook (site wide)
Generally called site.yml, above playbook directory
---
- include: control.yml
...

=====
ansible.cfg

- Located at /etc/ansibleansible.cfg
- If one is located in CWD then this will be used

Example file (overriding default inventory file location)
[defaults]
inventory = <relativeFilename>

=====
Tasks / Modules

We can run commands directly against hosts defined in the inventory file
Tasks return a status, commands which return non-zero (error) codes (like /bin/false) will cause ansible to report failures
Tasks generally return standard out capture
Good for debugging!
There are many, many modules - so check these first when a function is needed


//Example module commands (there are lots)
ansible -m command -a "<command>" <hosts> //Run a command on hosts (this is the default module if none spcecified)
ansible -m ping <hosts> //Ping hosts

//Some important ones (not in examples above generally)
- uri //Check an address, use register to save return content and then reference the id from the register to use the values
- fail //apply a when condition for failing a task (can apply to return from uri)
- shell //to run arbitrary shell commands

References
https://docs.ansible.com/ansible/2.9/modules/list_of_all_modules.html
https://en.wikipedia.org/wiki/Jinja_(template_engine)

=====
Types

Boolean -> These all work (in any case combo) - true, false, on, off, yes, no, y, n (but not t and f)

=====
Roles

Allows for re-use with some dynamic elements - Reuse of files, tasks, etc
Held in a roles directory
managed / shared with ansible-galaxy

ansible-galaxy init <role> //Create a role

Role files can be created to contain tasks, handlers, vars, templates, files, defaults - these will 'see each other; within the same role
<main>/roles/<roleName>/<type>/main.yaml
(vars should not be used really as this is an anti-pattern - specific values for a generic role)


The in playbooks simply reference the roles, the roles then pull all the bits:

Example (task in <main>/roles/control/tasks/main.yaml)
---
- name: install tools
  apt: name={{item}} state=present update_cache=yes
  with_items:
    - curl
    
Example (playbook):
---
  - hosts: control
    become: true
    roles:
      - control
    
=====
Facts (vars)

Information about the host gathered by ansible

ansible -m setup <host> to see what facts Ansible will gather

Theses can be referenced as dynamic values in playbooks
e.g. {{ ansible_eth0.ipv4.address }}

=====
Defaults (vars)

Define these in roles/<role>/defaults/main.yml
Example:
---
<var>: <value>
...


Theses can be referenced as dynamic values in playbooks
e.g. {{ ansible_eth0.ipv4.address }}

=====
Vars

Have an order of precedence...

role defaults
facts
variables in inventory
command line switches, vars in play, included vars, role vars...
connection variables (such as ansible_user)
extra vars (-e in cli)

vars can be added added directly in playbooks, group_vars, includes and many other places
Should a pattern and stick to it for an implementation

Can also add into roles (to reuse roles with different vars):
Example:
---
- hosts: database
  become: true
  roles:
    - { role: mysql, db_name: demo...}
    
with_dict can be used to defint dictionaries for reuse
Example:
---
sites:
  myapp:
    var: value
    ...
  myapp2
    ...

Example use (in a playbook):
      - name: configure nginx site
        template: src=templates/nginx.conf.j2 dest=/etc/nginx/sites-available/{{ item.key }} mode=0644 //i.e. the example in template section below
        with_dict: sites
        notify: restart nginx  
        
        {{ item.value.var }} //value    
        
        
group_vars - can be added at root (and auto apply to group as specified)
<root>/group_vars/<group>.yaml or <root>/group_vars/<group>/vars.yaml
Then reference in playbooks (can assign a var to a var - routing)

=====
Ansible Vault

Allows for encryption of secrets (with passphrase)
Only one passphrase allowed per execution

ansible-vault create <filename> //Asks for passphrase which is needed to decrypt - creates file in CWD
//Then provide the deets to be encrypted (in yaml)

When running playbooks which consume
--ask-vault-pass //One time ask on run
or put this in ~/.vault_pass.txt

=====
Optimisations / Debugging etc

gather_facts: false //Disable gather facts stage (in playbook at host level)

apt: update_cache=yes cache_valid_time=86400 //Update and reuse cache, can put in it's own playbook to run over all hosts - can then remove all the update_cache=yes from other tasks / playbooks

--limit <hosts> //Add to ansible-playbook command to limit to specific hosts - e.g. fixing a single host

tags: [ "<tag>" ] //Add tags (to a task or playbook)
ansible-playbook <file> --list-tags //to get all the tags for a playbook
ansible-playbook <file> --tags "<tags>" //Limit run to tags
ansible-playbook <file> --skip-tags "<tags>" //Inverse of above
//use the four pillars to separate here (best practice)

ansible-playbook <file> --step //Confirm each task before running
ansible-playbook <file> --list-tasks //List all the tasks in playbook
ansible-playbook <file> --start-at-task "<taskName>" //Start part way through a playbook

changed_when: false //Add to a task - ok or fail only status (for checks for example) can be an expression here instead
failed_when: //Also available to manage fail state
ignore_errors: true //Task fail does not fail playbook

Accelerated Mode //Legacy?
Pipelining //Better than AM - use this generally if you can (check docs)
(in ansible.cfg)
[ssh_connection]
pipelining: true

If one or more hosts fail, ansible will save these to a file and notify on run (so these can be re-ran separately)

- debug: <text or var=value> // print debug in a playbook (can use between tasks)