summaryrefslogtreecommitdiffhomepage
path: root/ansible.html.markdown
diff options
context:
space:
mode:
Diffstat (limited to 'ansible.html.markdown')
-rw-r--r--ansible.html.markdown755
1 files changed, 755 insertions, 0 deletions
diff --git a/ansible.html.markdown b/ansible.html.markdown
new file mode 100644
index 00000000..30dfba13
--- /dev/null
+++ b/ansible.html.markdown
@@ -0,0 +1,755 @@
+---
+category: tool
+tool: ansible
+contributors:
+ - ["Jakub Muszynski" , "http://github.com/sirkubax"]
+ - ["Pat Myron" , "https://github.com/patmyron"]
+ - ["Divay Prakash", "https://github.com/divayprakash"]
+filename: LearnAnsible.txt
+---
+
+## Introduction
+
+```yaml
+---
+"{{ Ansible }}" is an orchestration tool written in Python.
+...
+```
+
+Ansible is (one of many) orchestration tools. It allows you to control your
+environment (infrastructure and code) and automate the manual tasks.
+
+Ansible has great integration with multiple operating systems (even Windows)
+and some hardware (switches, Firewalls, etc). It has multiple tools that
+integrate with the cloud providers. Almost every noteworthy cloud provider is
+present in the ecosystem (AWS, Azure, Google, DigitalOcean, OVH, etc...).
+
+But ansible is way more! It provides execution plans, an API, library, and callbacks.
+
+### Main pros and cons
+
+#### Pros
+
+* It is an agent-less tool. In most scenarios, it uses ssh as a transport layer.
+In some way you can use it as 'bash on steroids'.
+* It is very easy to start. If you are familiar with the concept of ssh - you already
+know Ansible (ALMOST).
+* It executes 'as is' - other tools (salt, puppet, chef - might execute in
+different scenario than you would expect)
+* Documentation is at the world-class standard!
+* Writing your own modules and extensions is fairly easy.
+* Ansible AWX is the open source version of Ansible Tower we have been waiting
+for, which provides an excellent UI.
+
+#### Cons
+
+* It is an agent-less tool - every agent consumes up to 16MB ram - in some
+environments, it may be noticable amount.
+* It is agent-less - you have to verify your environment consistency
+'on-demand' - there is no built-in mechanism that would warn you about some
+change automatically (this can be achieved with reasonable effort)
+* Official GUI - Ansible Tower - is great but expensive.
+* There is no 'small enterprise' payment plan, however Ansible AWX is the free
+open source version we were all waiting for.
+
+#### Neutral
+
+Migration - Ansible <-> Salt is fairly easy - so if you would need an
+event-driven agent environment - it would be a good choice to start quick with
+Ansible, and convert to Salt when needed.
+
+#### Some concepts
+
+Ansible uses ssh or paramiko as a transport layer. In a way you can imagine
+that you are using a ssh with API to perform your action. The simplest way is
+to execute remote command in more controlled way (still using ssh).
+On the other hand - in advanced scope - you can wrap Ansible (use python Ansible
+code as a library) with your own Python scripts! It would act a
+bit like Fabric then.
+
+## Example
+
+An example playbook to install apache and configure log level
+
+```yaml
+---
+- hosts: apache
+
+ vars:
+ apache2_log_level: "warn"
+
+ handlers:
+ - name: restart apache
+ service:
+ name: apache2
+ state: restarted
+ enabled: True
+ notify:
+ - Wait for instances to listen on port 80
+ become: True
+
+ - name: reload apache
+ service:
+ name: apache2
+ state: reloaded
+ notify:
+ - Wait for instances to listen on port 80
+ become: True
+
+ - name: Wait for instances to listen on port 80
+ wait_for:
+ state: started
+ host: localhost
+ port: 80
+ timeout: 15
+ delay: 5
+
+ tasks:
+ - name: Update cache
+ apt:
+ update_cache: yes
+ cache_valid_time: 7200
+ become: True
+
+ - name: Install packages
+ apt:
+ name={{ item }}
+ with_items:
+ - apache2
+ - logrotate
+ notify:
+ - restart apache
+ become: True
+
+ - name: Configure apache2 log level
+ lineinfile:
+ dest: /etc/apache2/apache2.conf
+ line: "LogLevel {{ apache2_log_level }}"
+ regexp: "^LogLevel"
+ notify:
+ - reload apache
+ become: True
+...
+```
+
+## Installation
+
+```bash
+# Universal way
+$ pip install ansible
+
+# Debian, Ubuntu
+$ apt-get install ansible
+```
+
+* [Appendix A - How do I install ansible](#infrastructure-as-a-code)
+* [Additional Reading.](http://docs.ansible.com/ansible/latest/intro_installation.html)
+
+### Your first ansible command (shell execution)
+
+```bash
+# Command pings localhost (defined in default inventory: /etc/ansible/hosts)
+$ ansible -m ping localhost
+# You should see this output
+localhost | SUCCESS => {
+ "changed": false,
+ "ping": "pong"
+}
+```
+
+### Shell Commands
+
+There are few commands you should know about
+
+* `ansible` (to run modules in CLI)
+* `ansible-playbook` (to run playbooks)
+* `ansible-vault` (to manage secrets)
+* `ansible-galaxy` (to install roles from github/galaxy)
+
+### Module
+
+A program (usually python) that executes, does some work and returns proper
+JSON output. This program performs specialized task/action (like manage
+instances in the cloud, execute shell command). The simplest module is called
+`ping` - it just returns a JSON with `pong` message.
+
+Example of modules:
+
+* Module: `ping` - the simplest module that is useful to verify host connectivity
+* Module: `shell` - a module that executes a shell command on a specified host(s).
+
+
+```bash
+$ ansible -m ping all
+$ ansible -m shell -a 'date; whoami' localhost #hostname_or_a_group_name
+```
+
+* Module: `command` - executes a single command that will not be processed
+through the shell, so variables like `$HOME` or operands like ``|` `;`` will not
+work. The command module is more secure, because it will not be affected by the
+user’s environment. For more complex commands - use shell module.
+
+```bash
+$ ansible -m command -a 'date; whoami' # FAILURE
+$ ansible -m command -a 'date' all
+$ ansible -m command -a 'whoami' all
+```
+
+* Module: `file` - performs file operations (stat, link, dir, ...)
+* Module: `raw` - executes a low-down and dirty SSH command, not going through
+the module subsystem (useful to install python2.7)
+
+### Task
+
+Execution of a single Ansible **module** is called a **task**. The simplest
+module is called `ping` as you could see above.
+
+Another example of the module that allows you to execute a command remotely on
+multiple resources is called `shell`. See above how you were using them already.
+
+### Playbook
+
+**Execution plan** written in a form of script file(s) is called **playbook**.
+Playbooks consist of multiple elements -
+* a list (or group) of hosts that 'the play' is executed against
+* `task(s)` or `role(s)` that are going to be executed
+* multiple optional settings (like default variables, and way more)
+
+Playbook script language is YAML. You can think that playbook is very advanced
+CLI script that you are executing.
+
+#### Example of the playbook
+
+This example-playbook would execute (on all hosts defined in inventory) two tasks:
+* `ping` that would return message *pong*
+* `shell` that execute three commands and return the output to our terminal
+
+```yaml
+- hosts: all
+
+ tasks:
+ - name: "ping all"
+ ping:
+
+ - name: "execute a shell command"
+ shell: "date; whoami; df -h;"
+```
+
+Run the playbook with the command:
+
+```bash
+$ ansible-playbook path/name_of_the_playbook.yml
+```
+
+Note: Example playbook is explained in the next chapter: 'Roles'
+
+### More on ansible concept
+
+### Inventory
+
+An inventory is a set of objects or hosts, against which we are executing our
+playbooks or single tasks via shell commands. For these few minutes, let's
+assume that we are using the default ansible inventory (which in Debian based
+system is placed in `/etc/ansible/hosts`).
+
+```
+localhost
+
+[some_group]
+hostA.mydomain.com
+hostB.localdomain
+1.2.3.4
+
+[a_group_of_a_groups:children]
+some_group
+some_other_group
+```
+
+* [Additional Reading.](http://docs.ansible.com/ansible/latest/intro_inventory.html)
+
+### ansible-roles (a 'template-playbooks' with right structure)
+
+You already know that the tasks (modules) can be run via CLI. You also know the
+playbooks - the execution plans of multiple tasks (with variables and logic).
+
+A concept called `role` was introduced for parts of the code (playbooks) that
+should be reusable.
+
+**Role** is a structured way to manage your set of tasks, variables, handlers,
+default settings, and way more (meta, files, templates). Roles allow reusing
+the same parts of code in multiple playbooks (you can parametrize the role
+'further' during its execution). Its a great way to introduce `object oriented`
+management for your applications.
+
+Role can be included in your playbook (executed via your playbook).
+
+
+```yaml
+- hosts: all
+
+ tasks:
+ - name: "ping all"
+ ping:
+ - name: "execute a shell command"
+ shell: "date; whoami; df -h;"
+
+ roles:
+ - some_role
+ - { role: another_role, some_variable: 'learnxiny', tags: ['my_tag'] }
+
+ pre_tasks:
+ - name: some pre-task
+ shell: echo 'this task is the last, but would be executed before roles, and before tasks'
+```
+
+#### For remaining examples we would use additional repository
+This example installs ansible in `virtualenv` so it is independent from the system.
+You need to initialize it into your shell-context with the `source environment.sh`
+command.
+
+We are going to use this repository with examples: [https://github.com/sirkubax/ansible-for-learnXinYminutes](https://github.com/sirkubax/ansible-for-learnXinYminutes)
+
+```bash
+$ # The following example contains a shell-prompt to indicate the venv and relative path
+$ git clone git@github.com:sirkubax/ansible-for-learnXinYminutes.git
+user@host:~/$ cd ansible-for-learnXinYminutes
+user@host:~/ansible-for-learnXinYminutes$ source environment.sh
+$
+$ # First lets execute the simple_playbook.yml
+(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_playbook.yml
+```
+
+Run the playbook with roles example
+
+```bash
+$ source environment.sh
+$ # Now we would run the above playbook with roles
+(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_role.yml
+```
+
+#### Role directory structure
+
+```
+roles/
+ some_role/
+ defaults/ # contains default variables
+ files/ # for static files
+ templates/ # for jinja templates
+ tasks/ # tasks
+ handlers/ # handlers
+ vars/ # more variables (higher priority)
+ meta/ # meta - package (role) info
+```
+
+#### Role Handlers
+Handlers are tasks that can be triggered (notified) during execution of a
+playbook, but they execute at the very end of a playbook. It is the best way to
+restart a service, check if the application port is active (successful
+deployment criteria), etc.
+
+Get familiar with how you can use roles in the simple_apache_role example
+
+```
+playbooks/roles/simple_apache_role/
+├── tasks
+│   └── main.yml
+└── templates
+ └── main.yml
+```
+
+### ansible - variables
+
+Ansible is flexible - it has 21 levels of variable precedence.
+[read more](http://docs.ansible.com/ansible/latest/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable)
+For now you should know that CLI variables have the top priority.
+You should also know, that a nice way to pool some data is a **lookup**
+
+### Lookups
+Awesome tool to query data from various sources!!! Awesome!
+query from:
+* pipe (load shell command output into variable!)
+* file
+* stream
+* etcd
+* password management tools
+* url
+
+```bash
+# read playbooks/lookup.yml
+# then run
+(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/lookup.yml
+```
+
+You can use them in CLI too
+
+```yaml
+ansible -m shell -a 'echo "{{ my_variable }}"' -e 'my_variable="{{ lookup("pipe", "date") }}"' localhost
+ansible -m shell -a 'echo "{{ my_variable }}"' -e 'my_variable="{{ lookup("pipe", "hostname") }}"' all
+
+# Or use in playbook
+
+(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/lookup.yml
+```
+
+### Register and Conditional
+
+#### Register
+
+Another way to dynamically generate the variable content is the `register` command.
+`Register` is also useful to store an output of a task and use its value
+for executing further tasks.
+
+```
+(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/register_and_when.yml
+```
+
+```yaml
+---
+- hosts: localhost
+ tasks:
+ - name: check the system capacity
+ shell: df -h /
+ register: root_size
+
+ - name: debug root_size
+ debug:
+ msg: "{{ root_size }}"
+
+ - name: debug root_size return code
+ debug:
+ msg: "{{ root_size.rc }}"
+
+# when: example
+
+ - name: Print this message when return code of 'check the system capacity' was ok
+ debug:
+ msg: "{{ root_size.rc }}"
+ when: root_size.rc == 0
+...
+```
+
+#### Conditionals - when:
+
+You can define complex logic with Ansible and Jinja functions. Most common is
+usage of `when:`, with some variable (often dynamically generated in previous
+playbook steps with `register` or `lookup`)
+
+```yaml
+---
+- hosts: localhost
+ tasks:
+ - name: check the system capacity
+ shell: df -h /
+ when: some_variable in 'a string'
+ roles:
+ - { role: mid_nagios_probe, when: allow_nagios_probes }
+...
+```
+
+### ansible - tags, limit
+
+You should know about a way to increase efficiency by this simple functionality
+
+#### TAGS
+
+You can tag a task, role (and its tasks), include, etc, and then run only the
+tagged resources
+
+```
+ansible-playbook playbooks/simple_playbook.yml --tags=tagA,tag_other
+ansible-playbook playbooks/simple_playbook.yml -t tagA,tag_other
+
+There are special tags:
+ always
+
+--skip-tags can be used to exclude a block of code
+--list-tags to list available tags
+```
+
+[Read more](http://docs.ansible.com/ansible/latest/playbooks_tags.html)
+
+#### LIMIT
+
+You can limit an execution of your tasks to defined hosts
+
+```
+ansible-playbook playbooks/simple_playbook.yml --limit localhost
+
+--limit my_hostname
+--limit groupname
+--limit some_prefix*
+--limit hostname:group #JM
+```
+
+### Templates
+
+Templates are a powerful way to deliver some (partially) dynamic content.
+Ansible uses **Jinja2** language to describe the template.
+
+```
+Some static content
+
+{{ a_variable }}
+
+{% for item in loop_items %}
+ this line item is {{ item }}
+{% endfor %}
+```
+
+Jinja may have some limitations, but it is a powerful tool that you might like.
+
+Please examine this simple example that installs apache2 and generates
+index.html from the template
+"playbooks/roles/simple_apache_role/templates/index.html"
+
+```bash
+$ source environment.sh
+$ # Now we would run the above playbook with roles
+(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_role.yml --tags apache2
+```
+
+#### Jinja2 CLI
+
+You can use the jinja in the CLI too
+
+```bash
+ansible -m shell -a 'echo {{ my_variable }}' -e 'my_variable=something, playbook_parameter=twentytwo' localhost
+```
+
+In fact - jinja is used to template parts of the playbooks too
+
+```yaml
+# check part of this playbook: playbooks/roles/sys_debug/tasks/debug_time.yml
+- local_action: shell date +'%F %T'
+ register: ts
+ become: False
+ changed_when: False
+
+- name: Timestamp
+ debug: msg="{{ ts.stdout }}"
+ when: ts is defined and ts.stdout is defined
+ become: False
+```
+
+#### Jinja2 filters
+
+Jinja is powerful. It has many built-in useful functions.
+
+```
+# get first item of the list
+{{ some_list | first() }}
+# if variable is undefined - use default value
+{{ some_variable | default('default_value') }}
+```
+
+[Read More](http://docs.ansible.com/ansible/latest/playbooks_filters.html)
+
+### ansible-vault
+
+To maintain **infrastructure as code** you need to store secrets. Ansible
+provides a way to encrypt confidential files so you can store them in the
+repository, yet the files are decrypted on-the-fly during ansible execution.
+
+The best way to use it is to store the secret in some secure location, and
+configure ansible to use them during runtime.
+
+```bash
+# Try (this would fail)
+$ ansible-playbook playbooks/vault_example.yml
+
+$ echo some_very_very_long_secret > ~/.ssh/secure_located_file
+
+# in ansible.cfg set the path to your secret file
+$ vi ansible.cfg
+ ansible_vault_password_file = ~/.ssh/secure_located_file
+
+#or use env
+$ export ANSIBLE_VAULT_PASSWORD_FILE=~/.ssh/secure_located_file
+
+$ ansible-playbook playbooks/vault_example.yml
+
+ # encrypt the file
+$ ansible-vault encrypt path/somefile
+
+ # view the file
+$ ansible-vault view path/somefile
+
+ # check the file content:
+$ cat path/somefile
+
+ # decrypt the file
+$ ansible-vault decrypt path/somefile
+```
+
+### dynamic inventory
+
+You might like to know, that you can build your inventory dynamically.
+(For Ansible) inventory is just JSON with proper structure - if you can
+deliver that to ansible - anything is possible.
+
+You do not need to reinvent the wheel - there are plenty of ready to use
+inventory scripts for the most popular Cloud providers and a lot of in-house
+popular usecases.
+
+[AWS example](http://docs.ansible.com/ansible/latest/intro_dynamic_inventory.html#example-aws-ec2-external-inventory-script)
+
+```bash
+$ etc/inv/ec2.py --refresh
+$ ansible -m ping all -i etc/inv/ec2.py
+```
+
+[Read more](http://docs.ansible.com/ansible/latest/intro_dynamic_inventory.html)
+
+### ansible profiling - callback
+
+Playbook execution takes some time. It is OK. First make it run, then you may
+like to speed things up. Since ansible 2.x there is built-in callback for task
+execution profiling.
+
+```
+vi ansible.cfg
+# set this to:
+callback_whitelist = profile_tasks
+```
+
+### facts-cache and ansible-cmdb
+
+You can pull some information about your environment from another host.
+If the information does not change - you may consider using a facts_cache
+to speed things up.
+
+```
+vi ansible.cfg
+
+# if set to a persistent type (not 'memory', for example 'redis') fact values
+# from previous runs in Ansible will be stored. This may be useful when
+# wanting to use, for example, IP information from one group of servers
+# without having to talk to them in the same playbook run to get their
+# current IP information.
+fact_caching = jsonfile
+fact_caching_connection = ~/facts_cache
+fact_caching_timeout = 86400
+```
+
+I like to use `jsonfile` as my backend. It allows to use another project
+`ansible-cmdb` [(project on github)](https://github.com/fboender/ansible-cmdb) that generates a HTML page of your inventory
+resources. A nice 'free' addition!
+
+### Debugging ansible [chapter in progress]
+
+When your job fails - it is good to be effective with debugging.
+
+1. Increase verbosity by using multiple -v **[ -vvvvv]**
+2. If variable is undefined -
+`grep -R path_of_your_inventory -e missing_variable`
+3. If variable (dictionary or a list) is undefined -
+`grep -R path_of_your_inventory -e missing_variable`
+4. Jinja template debug
+5. Strange behaviour - try to run the code 'at the destination'
+
+### Infrastructure as code
+
+You already know, that ansible-vault allows you to store your confidential data
+along with your code. You can go further - and define your
+ansible installation and configuration as code.
+See `environment.sh` to learn how to install the ansible itself inside a
+`virtualenv` that is not attached to your operating system (can be changed by
+non-privileged user), and as additional benefit - upgrading version of ansible
+is as easy as installing new version in new virtualenv. What is more, you can
+have multiple versions of Ansible present at the same time.
+
+```bash
+# recreate ansible 2.x venv
+$ rm -rf venv2
+$ source environment2.sh
+
+# execute playbook
+(venv2)$ ansible-playbook playbooks/ansible1.9_playbook.yml # would fail - deprecated syntax
+
+# now lets install ansible 1.9.x next to ansible 2.x
+(venv2)$ deactivate
+$ source environment.1.9.sh
+
+# execute playbook
+(venv1.9)$ ansible-playbook playbooks/ansible1.9_playbook.yml # works!
+
+# please note that you have both venv1.9 and venv2 present - you need to (de)activate one - that is all
+```
+
+#### become-user, become
+
+In Ansible - to become `sudo` - use the `become` parameter. Use `become_user`
+to specify the username.
+
+```
+- name: Ensure the httpd service is running
+ service:
+ name: httpd
+ state: started
+ become: true
+```
+
+Note: You may like to execute Ansible with `--ask-sudo-pass` or add the user to
+sudoers file in order to allow non-supervised execution if you require 'admin'
+privilages.
+
+[Read more](http://docs.ansible.com/ansible/latest/become.html)
+
+## Tips and tricks
+
+#### --check -C
+
+Always make sure that your playbook can execute in 'dry run' mode (--check),
+and its execution is not declaring 'Changed' objects.
+
+#### --diff -D
+
+Diff is useful to see nice detail of the files changed.
+It compare 'in memory' the files like `diff -BbruN fileA fileB`.
+
+
+#### Execute hosts with 'regex'
+
+```bash
+ansible -m ping web*
+```
+
+#### Host groups can be joined, negated, etc
+
+```bash
+ansible -m ping web*:!backend:monitoring:&allow_change
+```
+
+#### Tagging
+
+You should tag some (not all) objects - a task in a playbook, all tasks
+included form a role, etc. It allows you to execute the chosen parts of the
+playbook.
+
+#### no_logs: True
+
+You may see, that some roles print a lot of output in verbose mode. There is
+also a debug module. This is the place where credentials may leak. Use `no_log`
+to hide the output.
+
+#### Debug module
+
+allows to print a value to the screen - use it!
+
+#### Register the output of a task
+
+You can register the output (stdout), rc (return code), stderr of a task with
+the `register` command.
+
+#### Conditionals: when:
+
+#### Loop: with, with\_items, with\_dict, with\_together
+
+[Read more](http://docs.ansible.com/ansible/latest/playbooks_conditionals.html)
+
+## Additional Resources
+
+* [Servers For Hackers: An Ansible Tutorial](https://serversforhackers.com/c/an-ansible-tutorial)
+* [A system administrator's guide to getting started with Ansible - FAST!](https://www.redhat.com/en/blog/system-administrators-guide-getting-started-ansible-fast)
+* [Ansible Tower](https://www.ansible.com/products/tower) - Ansible Tower provides a web UI, dashboard and rest interface to ansible.
+* [Ansible AWX](https://github.com/ansible/awx) - The Open Source version of Ansible Tower.