nagios

# Nagios Features

1. Infrastructure Monitoring
  a. Provided by: Nagios Core | Nagios XI (commercial offering)
  b. Uses standard protocols: ICMP, TCP, UDP, etc
  c. Host resources: Disk, CPU, RAM usage - NRPE add on
  d. Event handler for service restarts across platforms: linux, unix, windows, etc.
  e. Checks are performed every 5 minutes by default, unless overriden via HOST, HOSTGROUP, SERVICE, etc
  f. Active (Default, Nagios-initiated) and Passive (externally-initiated) checks are supported
  
2. Modular
  a. Nagios Core - Main monitoring engine
  b. Nagios XI - Commercial Monitoring engine
  c. Nagios V-Shell (Visual shell)
  d. Plugins
  e. etc
  
3. Object definitions
  a. Host - physical or virtual servers/infrastructure devices/IP-connected systems
  b. Services - ie HTTP, SMTP, DNS, etc
  c. Contacts - people who should be notified
  d. Commands -ie, command that changes DNS entries when a host is down
  e. Time Period - ie, when to check host-availabilty and/or when to notify contacts

4. Optional per-object (host) monitoring schedule - ie, every 1-minute, or 10-minutes
5. Schedule Templates (timeperiods_nagios2.cfg): 24x7, workhours, nonworkshours, never
6. Hosts can have parent/child dependecy relationship:
  a. switch (parent) and server (child), which is down parent or child?
7. Object inheritance - ie, top-level, master web-server object has attributes that are shared  with derived web server
   host objects
8. Service dependencies
9. Various notifications:
  a. SMS
  b. email
  c. custom - ie, write to file, log to DB, etc
10. Notification Schedules (notification period)
11. Notification States (notification_options) - governs which state generate notification: ie, 'd,u,r'
12. Parallel monitoring (service | host checks)
13. Open source and commercial
14. Extensible via addon - exchange.nagios.org
  a. ie Nagios Remote Plugin Executor (NRPE)
15. Schedule downtime of HOST(s) - avoids generating potentially costly false-positives
16. Split (Distributed) Configuration Files
  a. Main (nagios.cfg) - Nagios Daemon (Core)
  b. Objects - Hosts | HostGroups | Services | Contacts | Contact Groups | Commands | etc
  c. Resources - Data taht should NOT be read by the CGI - ie, credentials, macros
  d. CGI - Web Interface
17. Documentations
  a. Local - OS package - available within CGI
  b. Online
 
# Installations | Configurations Explorations
Features:
  1. Configures required items, including Apache HTTPD
  2. Auto-monitoring of localhost (127.0.0.1)

## Tasks
1. Install Nagios
  a. apt search nagios
  b. apt install nagios4
  c. dpkg -l | grep -i nagios
  d. dpkg -L nagios4
  
Note: Default CGI credentials are as follows
User = nagiosadmin
Passwd = admin

2. Explore Configuration: /etc/nagios4
  a. /etc/nagios
    - nagios.cfg - primary config file for nagios core
    Note: cfg_file - specifies config file to include: ie, hosts, services, contacts, etc
    Note: cfg_dir - specifies directory to include, containing config files (*.cfg) to process
    Note: cfg_* - directives can be used to reference distinct files and directories
    
  b. /etc/default/nagios
  c. /usr/share/plugins
  d. /etc/nagios-plugins - contains pre-rolled checks
    Note: Command names as well as object names should be unique
    
  e. /usr/lib/nagios/plugins - repository of various checks: ie, check_tcp, check_icmp
    Note: Commands are system-binaries that reside in: /usr/lib/nagios/plugins
    Note: Each command reveals usage information with --help
    
# ICMP (ping) monitoring
Features:
  1. ping monitoring using: /usr/lib/nagios/plugins/check_host -> check_icmp
  2. Predefined templates to handle most monitors:
    a. generic-host - checks target using ICMP with sensible defaults
    b. generic-host - can be inherited by HOST definition using: use generic-host
  3. Hosts and various objects are read from : /etc/nagios4/conf.d
  
Tasks:
  1. Monitor using ICMP: 192.168.1.20 (blue)
    a. use default: localhost.cfg as template
 ```
 # /etc/nagios4/conf.d/blue.cfg
 define host{
        use                     linux-server ; Name of host template to use
        host_name               blue
        alias                   blue server
        address                 192.168.1.20
        }
 ```
 Note: host_name - unique short name of the monitored host, used by host_group and service
 Note: CGI and Nagios do NOT update hosts, need to "service nagios4 reload"
 
  2. Alter per-host check-interval
    a. check_interval INTERVAL (num of minutes)
      - check_interval                  2
 
  3. Add Default Gateway: 192.168.1.1
    a. "netstat -rn" will display the gateway ip
    
# TCP | UDP Monitoring
Features:
  1. Directly monitors publically-accessible services published via: TCP/UDP
  2. Check commands: 'check_*' - can be run manually from command line to confirm results
  
Tasks:
  1. Monitor services (TCP/UDP) on 192.168.1.20
PORT     STATE SERVICE
22/tcp   open  ssh
8000/tcp open  http-alt

# /usr/lib/nagios/plugins/check_tcp -H 192.168.1.20 -p 8000
TCP OK - 0.001 second response time on 192.168.1.20 port 8000|time=0.000884s;;;0.000000;10.000000

# /usr/lib/nagios/plugins/check_ssh -H 192.168.1.20
SSH OK - OpenSSH_8.2p1 Ubuntu-4ubuntu0.1 (protocol 2.0) | time=0.023363s;;;0.000000;10.000000

2. Create http service for blue
# /usr/lib/nagios/plugins/check_http -H 192.168.1.20
HTTP OK: HTTP/1.1 200 OK - 854 bytes in 0.019 second response time |time=0.018839s;;;0.000000;10.000000 size=854B;;;0

 ```
 # /etc/nagios4/conf.d/blue.cfg
 define service{
        use                     generic-service ; Inherit values from a template
        host_name               blue
        service_description     http
        check_command           check_http
        }
```

# NRPE Nagios Remote Process Executor
Features:
  1. Ascertain local data:
    a. CPU
    b. RAM
    c. Current Users
    d. Disk Usage
  2. Supports:
    a. Direct (runs local checks via NRPE tunnel)
    b. Indirect (runs common checks via NRPE tunnel)
      ie, if Nagios Monitoring instance does NOT have access to target services, NRPE (Indirect) mode works well
  3. Restricted set of commands may run on target: /etc/nagios/nrpe.cfg
  4. Ability to execute remote plugins
  5. Supports SSL connections (default)
  6. Supports Clear-Text using: -n option with: check_nrpe*
  7. Uses TCP port 5666
  8. Implemented as 2 components
    a. Client - Runs on Monitoring Server (nagios web)
    b. Server - Runs on Target
    
Tasks:
  1. Install NRPE Server(192.168.1.20)
    a. apt install nagios-nrpe-server
    b. update 'allowed_hosts' on /etc/nagios/nrpe.cfg
    # grep 'check_users' /etc/nagios/nrpe.cfg 
    command[check_users]=/usr/lib/nagios/plugins/check_users -w 5 -c 10
    
  2. Install NRPE Client (nagios web)
    a. apt install nagios-nrpe-plugin
      a1. /etc/nagios-plugins/config/check_nrpe.cfg
      a2. /usr/lib/nagios/plugins/check_nrpe
      
  3. Run check manually to test from nagios web
    # /usr/lib/nagios/plugins/check_nrpe -H 192.168.1.20 -c check_users
    USERS OK - 2 users currently logged in |users=2;5;10;0
   
  4. Add to monitoring
 ```
 # /etc/nagios4/conf.d/blue.cfg
define service{
        use                     generic-service ; Inherit values from a template
        host_name               blue
        service_description     Current Users
        check_command           check_nrpe!check_users
        }

```

check_nrpe     !       check_users
----------  -------    ------------
command    delimeter   argument $ARG1$

# grep check_nrpe /etc/nagios-plugins/config/check_nrpe.cfg
	command_name	check_nrpe
	command_line	/usr/lib/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$


Note: Any plugin on the NRPE server can be monitored via NRPE
  a. Ensure plugin exists: /usr/lib/nagios/plugin/*
  b. Provision command in: /etc/nagios/nrpe.cfg
  c. call provisioned command from Nagios web Server
  
  
# Check Custom Strings
  1. Custom Service Check Scripts
  2. Returns various service states: OK, WARNING, CRITICAL, UNKNOWN
 
Tasks:
  1. Write a simple PHP script to perform: PHP -> MySQL test
  2. Create a webpage that will show if able to connect to db or not 
  3. Use: check_http to determine state of service (web application test of: PHP -> MySQL)
    a. /usr/lib/nagios/plugins/check_http -H www.linuxcbt.internal -r '1' -u '/webapptest.php'
    
  4. Create the command
```
# /etc/nagios-plugins/config/http.cfg
define command{
	command_name	check_webapp_php_mysql
	command_line	/usr/lib/nagios/plugins/check_http -H '$ARG1$' -r '$ARG2$' -u '$ARG3$'
	}
```
  
  5. Create the service and reference appropriate ARGs
```
# /etc/nagios4/conf.d/blue.cfg
define service{
        use                     generic-service ; Inherit values from a template
        host_name               blue
        service_description     Custom Web Test
        check_command           check_webapp_php_mysql!www.linuxcbt.internal!1!/webapptest.php
        }

```


# Object Groups
Features:
  1. Grouping of commonly classed objects: hosts, services, contacts, etc
  2. Visual representation in CGI
  3. Configuration Inheritance: ie, Services mapped to HostGroups
  4. Defaults are in: /etc/nagios4/con.d/hostsgroups*