----------------------------------------------------------------------- ASL_CHECK.SH V3.2.1 - 2002 by Volker Tanger ----------------------------------------------------------------------- This script monitors the health status of a remote Astaro firewall system (Astaro Security Linux = ASL) via SSH from the local NetSaint server - as active check results. Requirements: * Astaro firewall that has the public DSA key of root@NetSaintServer (or: netsaint@... depending on installation) installed at /home/login/.ssh/authorized_keys Please verify you can log in as this user! * NetSaint management server with SSH client and the usual shell tool suspects (sh/bash, fgrep, grep, cut) Updates will be available at http://www.wyae.de/software/aslcheck/ Please check there for updates prior to submitting patches! There is a user/developer mailing list available. To subscribe send a mail with "subscribe fwtools" as subject to minimalist@wyae.de For bug reports and suggestions or if you just want to talk to me please contact me at volker.tanger@wyae.de or fwtools@wyae.de ----------------------------------------------------------------------- Ressources monitored & Alert thresholds ----------------------------------------------------------------------- Local HDD partitions Warning: 70% full Critical: 90% full load (5 min average) Warning: 100% = 1.0 Critical: 500% = 5.0 log files size Warning: 100% above usual Critical: 500% above usual users logged in Warning: non-root interactive Critical: ROOT interactive number of processes Warning: 100 processes Critical: 500 processes zombie processes Warning: 1 "zombie" process Critical: 5 "zombie" processes Within the zombie monitoring there is an ugly workaround: AUA.BIN and CRON occasionally die - this is a known bug (= seen, but not understood why) which does not seem to hamper functionality. I want to know when they start running *really* wild, though. ----------------------------------------------------------------------- Configuration on the NetSaint management server ----------------------------------------------------------------------- There are a number of new commands defined. Place them e.g. at /usr/local/lib/netsaint/ and make sure they get execute permission. You will have to configure new "remote_..." commands in NetSaint. Please use the provided file "etc/netsaint/commands.cfg" for this. Either copy it directly or paste it into your current configuration file. The commands are defined as: # ASTARO Plugin Configuration for netsaint. # # Debian defaulr command definitions # for have been split into seperate files in /usr/share/netsaint/pluginconfig # and are automatically merged into /etc/netsaint/plugins-auto.cfg by # update-netsaint(8) # command[check_remoteping]=/usr/local/lib/netsaint/remote_ping.sh $ARG1$ $HOSTADDRESS$ command[check_remotedisk]=/usr/local/lib/netsaint/remote_disk.sh $ARG1$ $HOSTADDRESS$ command[check_remoteload]=/usr/local/lib/netsaint/remote_load.sh $HOSTADDRESS$ command[check_remoteprocess]=/usr/local/lib/netsaint/remote_process.sh $HOSTADDRESS$ command[check_remotezombie]=/usr/local/lib/netsaint/remote_zombie.sh $HOSTADDRESS$ command[check_remoteusers]=/usr/local/lib/netsaint/remote_users.sh $HOSTADDRESS$ command[check_asl_logs]=/usr/local/lib/netsaint/remote_asl_logs.sh $HOSTADDRESS$ After entering the host (here: "fw") into the host definition file (probably /etc/netsaint/hosts.cfg) and into a hostgroup you can copy&paste the following block directly into the file, below the two definitions mentioned above. Just replace the "fw"-part with your Astaro's name: service[ fw ]=rPING;0;24x7;3;1;1;admins;15;workhours;1;1;1;;check-host-alive service[ fw ]=hda1;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remotedisk!hda1 service[ fw ]=hda3;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remotedisk!hda3 service[ fw ]=hda5;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remotedisk!hda5 service[ fw ]=hda7;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remotedisk!hda7 service[ fw ]=hda8;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remotedisk!hda8 service[ fw ]=hda9;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remotedisk!hda9 service[ fw ]=load;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remoteload service[ fw ]=zombie;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remotezombie service[ fw ]=process;0;24x7;3;2;1;admins;15;workhours;1;1;1;;check_remoteprocess service[ fw ]=users;0;24x7;1;2;1;admins;15;workhours;1;1;1;;check_remoteusers service[ fw ]=asl_logs;0;24x7;1;2;1;admins;15;workhours;1;1;1;;check_asl_logs Ah, yes - don't forget to adapt your packet filter on the Astaro accordingly. ;-) ----------------------------------------------------------------------- Unusual Behaviour Detection ----------------------------------------------------------------------- While most tests can be applied to "any" unix server, one of the tests is special: "check_asl_logs". It compares the size of the logs to the "normal" rate usual during this hour. If the logs exceed the "normal" size, there is something different to yesterday - you should have a look... ----------------------------------------------------------------------- Known bugs and issues ----------------------------------------------------------------------- RISK - your management server: As you need to configure SSH access via DSA key authentication a compromised management server means for the attacker instant access to all the Astaros monitored by ASL_CHECK. Well, only LOGINUSER, but that's a (big!) first step. REQUIREMENT - ssh access from your management server: Your management server has to have direct SSH access to all the Astaros monitored. "BEATURE" - "Zombie" workaround ...see above UBD-"IDS" - only shows if there something REALLY is going wrong with your traffic. ----------------------------------------------------------------------- Design considerations ----------------------------------------------------------------------- After experimenting with various passive or semi-active bulk checks in versions 1.x and 2.x I came back to direct, plain, active checks. Providing passive check results prove to be unreliable. Reliability was bought with more CPU power needed due to the risen number of SSH connections for each check cycle. ----------------------------------------------------------------------- That's all - have fun! ----------------------------------------------------------------------- History ----------------------------------------------------------------------- Version 3.2.1 optimized getting the log file size - this way we have no more "expanding" traffic volume. Version 3.2.0 changed way of generating the "standard" log values. No separate cron job needed any more. Version 3.1.0 added "IDS" - chompare usual log file size against the current one. Version 3.0.3 added "asl_check_logs" and made the zombie check nicer. Version 3.0 now works as "generic" NetSaint plugin. No problems so far (except more load on the machines). Version 2.2 worked - kind of. Not very reliable, sometimes results were submitted as comments only. Strange, but not really reproducable. so dump it... Version 2.0 worked - if only one instance was running at once. Not really suited for monitoring a number of FWs. Version 1.0 ran locally on the Astaros and submitted the checks via NSCA transfer and cron job. This broke every time you got an update as the update would re-set the /etc/crontab needed. Plus you were not able to hide the management server behind MASQerading as all the other "normal" clients. ----------------------------------------------------------------------- Shortcut: Distributable under GPL ----------------------------------------------------------------------- Copyright (C) 2002 Volker Tanger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. or on their website http://www.gnu.org/copyleft/gpl.html -----------------------------------------------------------------------