Mastering Log Monitoring on Linux: Automate with Shell Scripts for Real-Time Alerts
Learn how to efficiently monitor logs on Linux systems using shell scripting. Detect critical events, automate alerts, and keep your infrastructure running smoothly.
Hey Mate! Welcome to my 92nd blog post. In this post, we are going to discuss monitoring using shell scrip for Linux machines
Here’s a Linux shell script that monitors key logs — journalctl, secure.log, and auth.log—for critical events or errors. It scans for new suspicious entries and sends an email alert to the IT admin if anything concerning is found.
Looking for Monitoring solutions, Please have a look Zabbix post here
le’s write a script!
The Monitoring Script (alert_monitor.sh)
#!/bin/bash
# Set variables
ADMIN_EMAIL="[email protected]" # Replace with your IT admin's email
TEMP_LOG="/tmp/critical_log_monitor.txt" # Temporary log file to store results
DATE=$(date +"%Y-%m-%d %H:%M:%S") # Current timestamp
# Function to search for errors and critical events in logs
function check_logs() {
echo "Critical Log Monitoring Report - $DATE" > "$TEMP_LOG"
echo "----------------------------------------" >> "$TEMP_LOG"
# Check journalctl for critical errors and alerts
echo "[journalctl - Critical Errors]" >> "$TEMP_LOG"
journalctl -p 3 -n 50 --no-pager >> "$TEMP_LOG" # Logs with priority 3 (Error)
echo "" >> "$TEMP_LOG"
# Check for failed login attempts or suspicious activities in secure.log
echo "[/var/log/secure - Failed Logins & Suspicious Activity]" >> "$TEMP_LOG"
grep -Ei 'failed|invalid|unauthorized|error' /var/log/secure | tail -n 50 >> "$TEMP_LOG"
echo "" >> "$TEMP_LOG"
# Check for failed authentications or privilege escalations in auth.log
echo "[/var/log/auth.log - Auth Issues & Privilege Escalation]" >> "$TEMP_LOG"
grep -Ei 'failure|authentication|sudo:.*(error|failure)' /var/log/auth.log | tail -n 50 >> "$TEMP_LOG"
echo "" >> "$TEMP_LOG"
# Display critical syslog events (replace with your log path if needed)
echo "[/var/log/syslog - Critical Events]" >> "$TEMP_LOG"
grep -Ei 'kernel:.*error|panic|segfault|critical' /var/log/syslog | tail -n 50 >> "$TEMP_LOG"
echo "" >> "$TEMP_LOG"
}
# Function to send email alerts
function send_alert() {
if [[ -s $TEMP_LOG ]]; then # Check if log file is not empty
mail -s "⚠️ Critical Log Alert - $DATE" "$ADMIN_EMAIL" < "$TEMP_LOG"
echo "Alert email sent to $ADMIN_EMAIL."
else
echo "No critical events found. No email sent."
fi
}
# Run log checks and send alerts
check_logs
send_alert
# Clean up temporary file
rm -f "$TEMP_LOG"Explanation of the Script:
Variables:
ADMIN_EMAIL: Replace this with the email address of your IT admin.TEMP_LOG: A temporary file to store log results.
Log Analysis:
journalctl: Scans for logs with priority3(Error).secure.log: Searches for failed login attempts or suspicious activity.auth.log: Checks for failed authentication attempts or privilege escalations.syslog: Looks for critical system events like kernel panics or segmentation faults.
Sending the Alert:
- If any critical log entry is found, it sends an email alert using the
mailcommand. - If no critical entries are found, it simply exits without sending an email.
Cleanup:
- After sending the alert, the temporary log file is deleted.
Setup Instructions:
Install mailutils (if not already installed):
sudo apt update && sudo apt install mailutilsGrant Execution Permission:
chmod +x alert_monitor.shSchedule the Script with Cron (Optional): Run the script periodically using cron. For example, to run every 10 minutes:
crontab -e
- Add the following line:
*/10 * * * * /path/to/alert_monitor.shHere is the guide for sending Email from the command Line
Testing the Script:
Simulate a failed login attempt to generate a log entry:
ssh invaliduser@localhostRun the script manually to ensure it captures the error:
./alert_monitor.sh
Final Notes:
This script provides a simple way to monitor key logs for issues and send email alerts to the admin in case of critical events. You can extend it to monitor additional logs or customize the grep patterns to match your environment.
Happy monitoring! 🚀
Here’s an improved version of the script that only filters critical events rather than displaying the last 50 lines. The filtering logic has been updated to extract critical logs dynamically, ensuring you focus only on relevant incidents.
#!/bin/bash
# Set variables
ADMIN_EMAIL="[email protected]" # Replace with your IT admin's email
TEMP_LOG="/tmp/critical_log_monitor.txt" # Temporary log file to store results
DATE=$(date +"%Y-%m-%d %H:%M:%S") # Current timestamp
# Function to check logs for critical events
function check_logs() {
echo "Critical Log Monitoring Report - $DATE" > "$TEMP_LOG"
echo "----------------------------------------" >> "$TEMP_LOG"
# Check journalctl for critical (priority 2) and error (priority 3) events
echo "[journalctl - Critical & Error Events]" >> "$TEMP_LOG"
journalctl -p 0..3 --no-pager --since "10 minutes ago" >> "$TEMP_LOG"
echo "" >> "$TEMP_LOG"
# Check secure.log for failed or suspicious logins
echo "[/var/log/secure - Critical Login Events]" >> "$TEMP_LOG"
grep -Ei 'critical|failed|unauthorized|error|invalid' /var/log/secure | grep "$(date '+%b %d')" >> "$TEMP_LOG"
echo "" >> "$TEMP_LOG"
# Check auth.log for authentication failures and privilege issues
echo "[/var/log/auth.log - Critical Auth Issues]" >> "$TEMP_LOG"
grep -Ei 'critical|failure|authentication error|sudo: .*error' /var/log/auth.log | grep "$(date '+%b %d')" >> "$TEMP_LOG"
echo "" >> "$TEMP_LOG"
# Check syslog for kernel panic, segmentation faults, or critical system errors
echo "[/var/log/syslog - Critical System Events]" >> "$TEMP_LOG"
grep -Ei 'kernel: .*error|panic|segfault|critical' /var/log/syslog | grep "$(date '+%b %d')" >> "$TEMP_LOG"
echo "" >> "$TEMP_LOG"
}
# Function to send email alerts
function send_alert() {
if [[ -s $TEMP_LOG ]]; then # Check if the log file contains any content
mail -s "⚠️ Critical Log Alert - $DATE" "$ADMIN_EMAIL" < "$TEMP_LOG"
echo "Alert email sent to $ADMIN_EMAIL."
else
echo "No critical events found. No email sent."
fi
}
# Run log checks and send alerts
check_logs
send_alert
# Clean up temporary file
rm -f "$TEMP_LOG"Explanation of Changes:
journalctl Filtering:
journalctl -p 0..3: Filters priority levels from 0 (Emergency) to 3 (Error).- Time filter:
--since "10 minutes ago"ensures only recent critical events are captured.
Secure and Auth Log Filters:
- The script only searches for today’s logs by matching the current date (
$(date '+%b %d')). - Error keywords such as “critical”, “failed”, “unauthorized”, and “invalid” are used to capture significant events.
Syslog Filter:
- Searches for critical events like kernel errors, segmentation faults, or panic events.
Efficient Email Alerting:
if [[ -s $TEMP_LOG ]]: Sends the email only if the log file is not empty.
If you like this article please give star on my GitHub
Follow for more: ✌️
Feel free to comment on your experience with our community
Publication: DevSecOp-Community
For Email Newsletter: Subscribe on Email Newsletter
For more updates: subscribe to this medium account.
Differ: https://differ.blog/@karthick-dkk
Github: https://github.com/karthick-dkk
LinkedIn: LinkedIn/karthick-dkk
