How to Set Up a Secure SSH Server on Linux: A Complete Guide

SSH: The Essential Tool for Remote Server Management

SSH (Secure Shell) is the standard protocol for securely accessing remote Linux servers. Whether you're managing a VPS, a home lab, or a fleet of cloud machines, knowing how to properly set up and secure SSH is a fundamental sysadmin skill.

Step 1: Install OpenSSH Server

Most Linux servers come with OpenSSH pre-installed. If not, install it with your distro's package manager:

Ubuntu/Debian: sudo apt install openssh-server
Fedora/RHEL: sudo dnf install openssh-server
Arch Linux: sudo pacman -S openssh

Enable and start the service:

sudo systemctl enable --now sshd

Verify it's running: sudo systemctl status sshd

Step 2: Configure the SSH Daemon

The main configuration file is /etc/ssh/sshd_config. Always back it up before editing:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

Key settings to configure:

Setting	Recommended Value	Purpose
`Port`	2222 (or custom)	Change from default 22 to reduce automated scans
`PermitRootLogin`	`no`	Prevent direct root login
`PasswordAuthentication`	`no`	Disable password login after setting up keys
`PubkeyAuthentication`	`yes`	Enable key-based login
`MaxAuthTries`	`3`	Limit login attempts per connection
`LoginGraceTime`	`30`	Seconds before unauthenticated connections timeout

After editing, test your configuration before restarting: sudo sshd -t

Step 3: Set Up Key-Based Authentication

Password-based SSH login is vulnerable to brute-force attacks. Key-based authentication is far more secure.

On your local machine, generate a key pair:

ssh-keygen -t ed25519 -C "your_email@example.com"

Use ed25519 — it's modern, fast, and more secure than RSA-2048.

Copy your public key to the server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@server-ip

Or manually append the public key to ~/.ssh/authorized_keys on the server.

Test key login, then disable passwords:

Once you confirm key-based login works, set PasswordAuthentication no in sshd_config and restart SSH:

sudo systemctl restart sshd

Step 4: Harden with Fail2Ban

Fail2Ban monitors log files and automatically bans IPs that show malicious behavior (like repeated failed logins).

sudo apt install fail2ban   # Ubuntu/Debian
sudo systemctl enable --now fail2ban

Create a local config file to override defaults:

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

In jail.local, set reasonable values for the SSH jail:

[sshd]
enabled = true
maxretry = 5
findtime = 600
bantime = 3600

This bans any IP that fails login 5 times within 10 minutes, for 1 hour.

Step 5: Configure the Firewall

Only allow SSH traffic on your chosen port. Using UFW (Ubuntu):

sudo ufw allow 2222/tcp
sudo ufw enable
sudo ufw status

If using firewalld (Fedora/RHEL):

sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --reload

Security Checklist

✅ Changed default port from 22
✅ Disabled root login
✅ Using key-based authentication only
✅ Disabled password authentication
✅ Installed and configured Fail2Ban
✅ Firewall allows only necessary ports
✅ Regularly audit ~/.ssh/authorized_keys

A properly secured SSH setup significantly reduces your attack surface. Combine these steps with regular system updates and you'll have a solid security baseline for any Linux server.