Linux Advanced Commands for Day-to-Day Activities for DevOps Engineers

Linux Advanced Commands for DevOps Engineers and SysAdmins -2

As a DevOps engineer, mastering advanced Linux commands is essential for streamlining workflows, automating tasks, and ensuring smooth operations in production environments.

This blog covers some of the most useful and advanced Linux commands that will boost your productivity and efficiency.

Let’s dive into key commands and see how they can be applied in day-to-day DevOps activities.

Basic Linux Commands

• tmux - Terminal Multiplexing for Efficient Sessions

# Running background jobs while monitoring logs simultaneously in separate panes.
tmux new -s devops_session

# Reattaches a previously detached session,
tmux attach -t devops_session

• awk - Data Extraction and Reporting Tool

# Extract specific fields from log 
awk '{print $1, $3}' /var/log/syslog

# searches for "error" in the syslog file and counts
awk '/error/ {count++} END {print count " errors found"}' /var/log/syslog

• sed - Stream Editing at Scale

# Updating configuration files
sed -i 's/oldtext/newtext/g' filename.txt

# removes commented and blank lines from a configuration file
sed -e '/^#/d' -e '/^$/d' config.txt

• rsync - Fast and Efficient File Synchronization

# Syncing files between one server to another server
rsync -avzh /source/path/ /destination/path/

# excludes specific directories (like temp/) from being synchronized
rsync -avzh --exclude 'temp/' /source/ /destination/

• lsof - List Open Files and Diagnose System Issues

 Identifying which process is using a specific port 8080
lsof -i :8080

# find files that are still held by a process after being deleted
lsof | grep deleted

• netstat - Network Diagnostics and Monitoring

# Checking open ports
netstat -tuln

# all active connections to port 80
netstat -antp | grep 80

• strace - Troubleshoot Programs in Real-Time

# Debugging a process
strace -p <PID>

# logs all system calls to a file for later analysis
strace -o trace_output.txt -p <PID>

• htop - Interactive Process Monitoring

# Quickly identifying resource-hogging processes
htop

Advanced Commands for File Management

1. find - Advanced File Search

Command Example:

find /var/www -type f -name "*.log" -mtime +30 -delete

Explanation: The find command searches for files in a directory hierarchy. With this advanced usage, you can find log files older than 30 days and delete them to free up disk space.

• Use Case: Cleaning up old logs or files that clutter the system.

Advanced Option:

find /etc -type f -exec chmod 644 {} \;

This changes file permissions recursively for all files in the /etc directory.

2. xargs - Building Commands from Standard Input

Command Example:

find . -name "*.log" | xargs grep "ERROR"

Explanation: xargs takes input from a command like find and passes it as arguments to another command. It’s a powerful tool when dealing with bulk file operations.

• Use Case: Searching through multiple files for specific patterns, useful for log analysis.

cat filelist.txt | xargs -I {} cp {} /backup/

Copies of files listed in filelist.txt to the /backup/ directory.

3. tar - Archiving Files

Command Example:

tar -cvzf backup.tar.gz /var/www

Explanation: tar is used for creating archives. The example compresses the /var/www directory into a .tar.gz archive.

• Use Case: Creating backups of important directories or files.

tar --exclude='/var/www/temp' -cvzf backup.tar.gz /var/www

Excludes specific directories from the archive to avoid unnecessary files.

Advanced Commands for Services

1. systemctl - Managing System Services

Command Example:

systemctl status nginx

Explanation: systemctl is the core tool to manage system services. It’s used to start, stop, restart, or view the status of services running on your Linux system.

• Use Case: Checking the status of web services like Nginx or Apache.

systemctl enable --now docker

Enables the Docker service at boot and starts it immediately.

2. journalctl - View Logs from Systemd Services

Command Example:

journalctl -u nginx.service --since "1 hour ago"

Explanation: journalctl provides an easy way to view logs from systemd services. In this example, it shows logs from the Nginx service over the last hour.

• Use Case: Diagnosing service errors or reviewing past events.

journalctl -xe

Shows the most recent critical log entries with detailed explanations, helping with immediate troubleshooting.

Advanced Commands for Security

1. iptables - Configuring Firewall Rules

Command Example:

iptables -A INPUT -p tcp --dport 22 -j ACCEPT

Explanation: iptables manages firewall rules on Linux systems. This example allows SSH connections on port 22.

• Use Case: Restricting access to certain services or securing server entry points.

iptables -L --line-numbers

Displays all current rules with line numbers for easy modification or deletion.

2. fail2ban - Preventing Brute Force Attacks

Command Example:

fail2ban-client status

Explanation: fail2ban is a tool that scans log files for repeated failed login attempts and blocks the offending IP addresses.

• Use Case: Enhancing SSH or web server security by blocking brute-force attacks.

fail2ban-client set sshd banip 192.168.1.100

Manually bans a specific IP address from SSH access.

3. gpg - File Encryption and Decryption

Command Example:

gpg -c secretfile.txt

Explanation: gpg (GNU Privacy Guard) allows you to encrypt and decrypt files. This example encrypts secretfile.txt.

• Use Case: Securing sensitive files before transferring them across servers.

gpg --decrypt secretfile.txt.gpg

Decrypts the encrypted file using the appropriate key.

Advanced Commands for Process Control

1. kill - Terminating Processes

Command Example:

kill -9 1234

Explanation: kill sends signals to processes, usually to terminate them. The -9 option forcefully kills the process.

• Use Case: Stopping runaway processes or those consuming too many resources.

killall -u username

Kills all processes running under a specific user.

2. nice and renice - Priority Adjustment for Processes

Command Example:

nice -n 10 ./long_running_script.sh

Explanation: nice changes the priority of a process when it’s started, which helps balance CPU resources for other tasks.

• Use Case: Running resource-intensive scripts without impacting system performance.

renice 5 -p 1234

Changes the priority of an already running process with PID 1234.

Advanced Commands for System Monitoring

1. iostat - Disk I/O Statistics

Command Example:

iostat -x 1 10

Explanation: iostat provides detailed statistics on disk I/O, helping you identify potential bottlenecks in disk performance.

• Use Case: Monitoring disk performance during heavy workloads.

2. vmstat - Virtual Memory Statistics

Command Example:

vmstat 2 5

Explanation: vmstat reports virtual memory, CPU, and disk I/O usage. The example reports every 2 seconds for a total of 5 iterations.

• Use Case: Monitoring system health and resource usage over time.

Advanced Miscellaneous Commands

1. curl - Interacting with Web APIs

Command Example:

curl -X POST -H "Content-Type: application/json" -d '{"username":"admin","password":"pass"}' http://example.com/api/login

Explanation: curl is a command-line tool for transferring data with URLs. It’s particularly useful for interacting with APIs, automating web requests, or downloading files.

• Use Case: Testing REST APIs or downloading resources from the web.

curl -O http://example.com/file.zip

Download a file from a URL directly to your server.

2. watch - Repeating Commands at Intervals

Command Example:

watch -n 5 df -h

Explanation: watch runs a command repeatedly at specified intervals. In this example, it runs df -h every 5 seconds to monitor disk space in real-time.

• Use Case: Live monitoring of system resources or process statuses.

This extended list of advanced Linux commands for DevOps engineers provides a solid toolkit for managing files, services, security, processes, and system monitoring. Each command can be leveraged to optimize workflows, improve security, and enhance operational efficiency.

Follow for more: ✌️

Publication: DevSecOp-Community

Follow my Medium Account (To get valuable information)

For Email Notify: Subscribe on Email Newsletter

For more updates: subscribe to this medium account.

LinkedIn: LinkedIn/karthick-dkk