Operating System Setup and Hardening
Learning Objectives
Configure a fresh Ubuntu server with security-hardened baseline settings
Implement SSH hardening including key-only authentication and brute-force protection
Design firewall rules that protect the validator while allowing necessary network communication
Configure NTP synchronization critical for consensus participation
Apply kernel and system tuning appropriate for high-performance network applications
Your validator's security is only as strong as its weakest layer. A perfectly configured rippled instance running on a poorly secured operating system is vulnerable. Attackers target the path of least resistance—often SSH brute-force attacks, unpatched vulnerabilities, or misconfigured services rather than direct protocol attacks.
This lesson establishes a hardened foundation before we install any blockchain software. The practices here aren't specific to XRPL—they're standard server hardening that applies to any production system handling sensitive operations. However, validators have additional considerations: they must maintain network connectivity to specific ports while rejecting everything else, they require precise time synchronization, and they benefit from network performance tuning.
If you've hardened production servers before, much of this will be familiar. If you haven't, pay close attention—these practices will serve you far beyond validator operation.
Prerequisite: You should have a provisioned server from your chosen provider with fresh Ubuntu 22.04 or 24.04 LTS installed. The examples use Ubuntu 24.04 LTS (Noble Numbat), currently the recommended platform.
Most providers give you root access via password or SSH key. Your first task is connecting and assessing the starting state.
# Connect to your new server (replace with your IP)
ssh root@YOUR_SERVER_IP
If using password authentication (change immediately):
You'll be prompted for the root password
If provider set up your SSH key (better):
ssh -i ~/.ssh/your_key root@YOUR_SERVER_IP
```
First commands after connection:
# Update package lists
apt update
# Upgrade all packages (critical security step)
apt upgrade -y
# Check Ubuntu version
lsb_release -a
# Check current users
cat /etc/passwd | grep -v nologin | grep -v false
# Check listening services
ss -tulpn
Running as root is dangerous—a single mistake has maximum impact. Create a dedicated user for daily operations.
# Create user for validator operations
# Replace 'validator' with your preferred username
adduser validator
Follow prompts for password (use a strong password)
Other fields can be left blank
Add user to sudo group for administrative tasks
usermod -aG sudo validator
Verify group membership
groups validator
Should show: validator : validator sudo
Configure SSH key authentication for the new user before disabling root login.
# Switch to the new user
su - validator
Create SSH directory with correct permissions
mkdir -p ~/.ssh
chmod 700 ~/.ssh
Create authorized_keys file
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
Add your public key (paste your public key here)
nano ~/.ssh/authorized_keys
Paste your public key (ssh-rsa AAAA... or ssh-ed25519 AAAA...)
Save and exit (Ctrl+X, Y, Enter)
Verify permissions
ls -la ~/.ssh/
Should show:
drwx------ .ssh
-rw------- authorized_keys
Test the new user login before proceeding:
# From your LOCAL machine, open a NEW terminal:
ssh validator@YOUR_SERVER_IP
# If using a specific key:
ssh -i ~/.ssh/your_key validator@YOUR_SERVER_IP
# Verify sudo works:
sudo whoami
# Should output: root
Critical: Do not proceed until you can successfully log in as the new user with sudo access. Locking yourself out at this stage requires provider console access to recover.
SSH is the primary attack surface for most servers. Proper configuration prevents the vast majority of unauthorized access attempts.
Edit the SSH daemon configuration:
# Backup original configuration
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup
# Edit SSH configuration
sudo nano /etc/ssh/sshd_config
Recommended settings (find and modify these lines):
# Disable root login via SSH
PermitRootLogin no
# Disable password authentication (key only)
PasswordAuthentication no
# Disable empty passwords
PermitEmptyPasswords no
# Use only SSH Protocol 2
Protocol 2
# Limit authentication attempts
MaxAuthTries 3
# Set login grace time (seconds to authenticate)
LoginGraceTime 30
# Disable X11 forwarding (not needed for server)
X11Forwarding no
# Disable TCP forwarding unless specifically needed
AllowTcpForwarding no
# Only allow specific user(s)
AllowUsers validator
# Set idle timeout (optional, prevents abandoned sessions)
ClientAliveInterval 300
ClientAliveCountMax 2
Optional: Change SSH port (security through obscurity)
# Change from default port 22
# Choose a port between 1024-65535
Port 2222Note: Changing the SSH port reduces automated scanning noise but isn't true security. If you change it, remember the new port for all future connections.
Apply changes:
# Test configuration syntax
sudo sshd -t
# If no errors, restart SSH service
sudo systemctl restart sshd
# Verify SSH is running
sudo systemctl status sshd
Critical: Keep your current session open. Open a NEW terminal and verify you can connect:
# If you changed the port:
ssh -p 2222 validator@YOUR_SERVER_IP
# If using default port:
ssh validator@YOUR_SERVER_IP
Only close your original session after confirming the new connection works.
Fail2ban automatically blocks IP addresses that show malicious signs, like repeated failed login attempts.
# Install fail2ban
sudo apt install fail2ban -y
Create local configuration (don't edit jail.conf directly)
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
Edit local configuration
sudo nano /etc/fail2ban/jail.local
```
Recommended settings:
[DEFAULT]
# Ban duration (10 minutes)
bantime = 10m
# Time window for counting failures
findtime = 10m
# Number of failures before ban
maxretry = 5
# Email notifications (optional)
# destemail = [email protected]
# sendername = Fail2Ban
# action = %(action_mwl)s
[sshd]
enabled = true
port = ssh
# If you changed SSH port:
# port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 1h
Enable and start fail2ban:
# Enable on boot
sudo systemctl enable fail2ban
# Start service
sudo systemctl start fail2ban
# Check status
sudo fail2ban-client status
# Check SSH jail specifically
sudo fail2ban-client status sshd
# View recent authentication attempts
sudo grep "sshd" /var/log/auth.log | tail -20
Check for failed attempts
sudo grep "Failed password" /var/log/auth.log | tail -10
View currently banned IPs
sudo fail2ban-client status sshd
```
A properly configured firewall is essential for validator security. We'll use UFW (Uncomplicated Firewall), Ubuntu's standard firewall interface.
# Install UFW (usually pre-installed)
sudo apt install ufw -y
Check current status
sudo ufw status
Set default policies
Deny all incoming (whitelist approach)
sudo ufw default deny incoming
Allow all outgoing (validator needs to connect out)
sudo ufw default allow outgoing
```
# Allow SSH (use your port if changed)
sudo ufw allow ssh
# Or if using custom port:
# sudo ufw allow 2222/tcp
Allow rippled peer port (required for validator)
Default is 51235
sudo ufw allow 51235/tcp
DO NOT allow admin RPC port (5005) from outside
It should only be accessible locally
If you need WebSocket API access (optional, not recommended for validators)
sudo ufw allow 6006/tcp
View rules before enabling
sudo ufw show added
```
# Enable UFW (will prompt for confirmation)
sudo ufw enable
Check status with rules
sudo ufw status verbose
Should show:
Status: active
To Action From
-- ------ ----
22/tcp (or your port) ALLOW Anywhere
51235/tcp ALLOW Anywhere
Validator Firewall Rules:
- SSH (your chosen port) - Administrative access
- 51235/tcp (peer port) - Required for P2P communication
- 5005/tcp (admin RPC) - Administrative commands, local only
- 5006/tcp (public WebSocket) - Unless serving public API
- Rate limiting on peer port (advanced)
- Geographic restrictions if applicable
- VPN requirement for admin access (enterprise)
Example advanced rule (rate limit peer connections):
sudo ufw limit 51235/tcp comment 'Rate limit peer connections'
Accurate time is essential for consensus participation. Validators with clock skew may have votes rejected or excluded from consensus rounds.
Why time matters for validators:
- Ledgers close every 3-5 seconds
- Validation messages are timestamped
- Significant clock skew = rejected validations
- Validators with bad time appear unreliable
- Within 1 second of accurate time
- Sub-second preferred
- Consistent sync (not jumping around)
- Standard solution for time sync
- Multiple time servers for redundancy
- Typically achieves <50ms accuracy
- Essential for any validator
Ubuntu uses systemd-timesyncd by default. For most validators, this is sufficient.
# Check current time sync status
timedatectl status
Should show:
System clock synchronized: yes
NTP service: active
If NTP is not active:
sudo timedatectl set-ntp on
Check timesyncd status
systemctl status systemd-timesyncd
```
Configure time servers:
# Edit timesyncd configuration
sudo nano /etc/systemd/timesyncd.conf
# Add reliable NTP servers:
[Time]
NTP=time.google.com time.cloudflare.com
FallbackNTP=ntp.ubuntu.com pool.ntp.org
# Restart timesyncd
sudo systemctl restart systemd-timesyncd
# Verify sync
timedatectl timesync-status
For operators wanting more precise time sync or more control:
# Install chrony (will replace timesyncd)
sudo apt install chrony -y
# Edit chrony configuration
sudo nano /etc/chrony/chrony.conf
# Add at the top (comment out existing pool lines):
server time.google.com iburst
server time.cloudflare.com iburst
server time.facebook.com iburst
pool pool.ntp.org iburst
# Restart chrony
sudo systemctl restart chrony
# Check synchronization
chronyc tracking
# View sources
chronyc sources -v
# Check system time vs NTP
timedatectl
For chrony users:
chronyc tracking
Key metrics:
- "System clock synchronized: yes"
- Offset should be < 100ms (< 0.1 seconds)
- Stratum should be 2-4 (not too high)
Quick test against external source
date && curl -s http://worldtimeapi.org/api/ip" target="_blank" rel="noopener noreferrer" class="text-cyan-400 hover:text-cyan-300 underline hover:no-underline transition-colors inline-flex items-center gap-1">http://worldtimeapi.org/api/ip">http://worldtimeapi.org/api/ip | jq .datetime
```
High-performance network applications benefit from system tuning. These settings optimize Linux for rippled's network-intensive operations.
# Create sysctl configuration for validator
sudo nano /etc/sysctl.d/99-validator.confAdd the following settings:
# Network buffer sizes
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 1048576
net.core.wmem_default = 1048576
# TCP buffer sizes
net.ipv4.tcp_rmem = 4096 1048576 16777216
net.ipv4.tcp_wmem = 4096 1048576 16777216
# Connection handling
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535
# TCP optimizations
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
# Keep-alive settings
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15
# File descriptor limits
fs.file-max = 2097152
# Memory settings
vm.swappiness = 10
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2
Apply settings:
# Apply immediately
sudo sysctl -p /etc/sysctl.d/99-validator.conf
# Verify settings
sysctl net.core.rmem_max
sysctl vm.swappiness
rippled opens many network connections. Increase file descriptor limits:
# Edit limits configuration
sudo nano /etc/security/limits.d/99-validator.confAdd:
# Limits for validator user
validator soft nofile 65535
validator hard nofile 65535
validator soft nproc 65535
validator hard nproc 65535
# Root limits (for service management)
root soft nofile 65535
root hard nofile 65535
Also update systemd defaults:
sudo nano /etc/systemd/system.confFind and modify (or add):
DefaultLimitNOFILE=65535Apply changes:
# Reload systemd
sudo systemctl daemon-reload
# Log out and back in for limits to apply
exit
# ... reconnect ...
# Verify limits
ulimit -n
# Should show 65535
Keep the system patched automatically for security updates:
# Install unattended-upgrades (usually pre-installed)
sudo apt install unattended-upgrades -y
# Configure automatic updates
sudo dpkg-reconfigure -plow unattended-upgrades
# Select "Yes" when prompted
# Verify configuration
cat /etc/apt/apt.conf.d/20auto-upgrades
# Should show:
# APT::Periodic::Update-Package-Lists "1";
# APT::Periodic::Unattended-Upgrade "1";
# Check what will be auto-updated
cat /etc/apt/apt.conf.d/50unattended-upgrades | grep -v "^//"
Reduce attack surface by disabling services you don't need:
# List all running services
systemctl list-units --type=service --state=running
# Common services to disable on a validator:
# (Be careful - verify you don't need them)
# If not using printer services:
sudo systemctl disable cups-browsed 2>/dev/null
sudo systemctl stop cups-browsed 2>/dev/null
# If not using Avahi (mDNS):
sudo systemctl disable avahi-daemon 2>/dev/null
sudo systemctl stop avahi-daemon 2>/dev/null
# If not using Bluetooth:
sudo systemctl disable bluetooth 2>/dev/null
sudo systemctl stop bluetooth 2>/dev/null
Run through this checklist to verify hardening:
# 1. SSH Configuration
sudo grep -E "^(PermitRootLogin|PasswordAuthentication|Port)" /etc/ssh/sshd_config
# Expected: PermitRootLogin no, PasswordAuthentication no
# 2. Firewall Status
sudo ufw status
# Expected: Active with only necessary ports
# 3. Fail2ban Status
sudo fail2ban-client status sshd
# Expected: Active, monitoring SSH
# 4. Time Sync
timedatectl | grep synchronized
# Expected: System clock synchronized: yes
# 5. File Descriptor Limits
ulimit -n
# Expected: 65535
# 6. Kernel Parameters
sysctl vm.swappiness
# Expected: 10
# 7. Automatic Updates
systemctl status unattended-upgrades
# Expected: Active
Optional but recommended—run a basic security scan:
# Install Lynis security auditing tool
sudo apt install lynis -y
# Run security audit
sudo lynis audit system
# Review the report
# Focus on "Warnings" and "Suggestions"
# Not all suggestions apply to every use case
Create a configuration record:
# Create documentation directory
mkdir -p ~/validator-docs
# Record system information
uname -a > ~/validator-docs/system-info.txt
lsb_release -a >> ~/validator-docs/system-info.txt
# Record network configuration
ip addr > ~/validator-docs/network-config.txt
sudo ufw status verbose >> ~/validator-docs/network-config.txt
# Record SSH configuration (redact sensitive details)
sudo grep -E "^[^#]" /etc/ssh/sshd_config > ~/validator-docs/ssh-config.txt
# Record sysctl settings
sysctl -a 2>/dev/null | grep -E "(net.core|net.ipv4.tcp|vm.)" > ~/validator-docs/sysctl-settings.txt
With the operating system hardened, prepare for rippled installation:
# Verify Ubuntu version (22.04 or 24.04)
lsb_release -a
Verify available disk space
df -h
Verify memory
free -h
Verify CPU
lscpu | grep -E "(Model name|CPU(s))"
Verify time sync
timedatectl | grep synchronized
Verify firewall allows peer port
sudo ufw status | grep 51235
```
# rippled will create directories, but pre-create with correct ownership:
sudo mkdir -p /var/lib/rippled/db
sudo mkdir -p /var/log/rippled
These will be owned by the rippled user created during installation
For now, verify they exist:
ls -la /var/lib/rippled/
ls -la /var/log/rippled/
```
Apply all changes with a clean reboot:
# Reboot the system
sudo reboot
# Wait 1-2 minutes, then reconnect
ssh validator@YOUR_SERVER_IP
# Verify everything came up correctly:
sudo ufw status
sudo systemctl status fail2ban
timedatectl
ulimit -n
✅ SSH key-only authentication prevents brute-force attacks - Password authentication is the primary attack vector for most compromised servers; eliminating it dramatically improves security
✅ Fail2ban significantly reduces attack noise - Automated blocking of repeat offenders prevents resource exhaustion and log pollution from constant scanning
✅ NTP accuracy is critical for validators - Validators with clock skew have validation messages rejected; time sync is non-negotiable
✅ Default firewall policies should deny incoming - Whitelist approach ensures only explicitly allowed traffic reaches the server
⚠️ Optimal sysctl values vary by workload - The provided settings are good starting points; optimal values depend on your specific network conditions and hardware
⚠️ Security scanning tool recommendations - Lynis and similar tools provide suggestions, but not all apply to every use case; use judgment
⚠️ Custom SSH port effectiveness - Reduces automated scanning but doesn't provide real security; useful but shouldn't be relied upon
📌 Locking yourself out during SSH configuration - Always test new SSH configuration before closing your existing session
📌 Firewall misconfiguration blocking legitimate traffic - Verify peer port (51235) is open before installing rippled
📌 Disabling services you actually need - Verify services aren't required before disabling; some providers use specific services for management
📌 Skipping automatic updates - Unpatched systems are the most common compromise vector; automatic security updates are essential
The hardening steps in this lesson are standard server security practices—nothing exotic or XRPL-specific. However, they're essential because a compromised server means a compromised validator. The time investment here (2-3 hours for careful implementation) prevents future security incidents that could damage your reputation in the validator community.
Most server compromises occur through predictable vectors: weak SSH authentication, unpatched vulnerabilities, and unnecessary exposed services. Addressing these basics provides strong security for most threat models without exotic complexity.
Assignment: Complete the hardening process on your provisioned server and document evidence of each step.
Requirements:
Screenshot of
grep -E "^(PermitRootLogin|PasswordAuthentication)" /etc/ssh/sshd_configshowing correct settingsScreenshot of
sudo fail2ban-client status sshdshowing active jailDocument your SSH port (default 22 or custom)
Verify non-root user with sudo access
Screenshot of
sudo ufw status verboseshowing enabled with correct rulesDocument each allowed port and justification
Confirm default deny incoming policy
Verify peer port 51235 is allowed
Screenshot of
timedatectlshowing synchronized clockDocument configured NTP servers
Verify time accuracy (< 100ms offset)
Screenshot of
ulimit -nshowing 65535Document sysctl changes in /etc/sysctl.d/99-validator.conf
Verify vm.swappiness = 10
Run and document Lynis security scan results
Screenshot of running services (
systemctl list-units --type=service --state=running)Confirm automatic security updates enabled
Document any warnings and how you addressed them
PDF or Markdown document with screenshots
Include command outputs, not just claims
Note any deviations from lesson guidance with rationale
Complete SSH hardening with evidence (25%)
Correct firewall configuration (25%)
Verified time synchronization (20%)
System tuning applied (15%)
Overall security verification (15%)
Time investment: 2-4 hours
Value: A hardened server foundation that protects your validator investment
1. SSH Security (Tests Practical Knowledge):
After configuring SSH with PermitRootLogin no and PasswordAuthentication no, what must you verify BEFORE closing your current session?
A) That the firewall is enabled
B) That you can successfully connect in a new terminal window using your new user account with SSH key
C) That fail2ban is running
D) That NTP is synchronized
Correct Answer: B
Explanation: If you close your session without verifying you can connect with the new configuration, you may lock yourself out of the server entirely. Always test SSH changes in a new terminal while keeping the original session open as backup. Firewall, fail2ban, and NTP are important but won't help if you can't log in.
2. Firewall Configuration (Tests Security Understanding):
Why should the rippled admin RPC port (5005) NOT be exposed to the internet?
A) It uses too much bandwidth
B) It allows administrative commands that could compromise the validator if accessed by attackers
C) It's not needed for validator operation
D) Ripple's documentation prohibits it
Correct Answer: B
Explanation: The admin RPC port allows commands like stopping the server, submitting transactions, and accessing sensitive configuration. If exposed to the internet, attackers could disrupt your validator or potentially compromise it further. This port should only be accessible locally (127.0.0.1) or via VPN/SSH tunnel for legitimate administration.
3. Time Synchronization (Tests Consensus Understanding):
What happens if a validator has significant clock skew (several seconds off from accurate time)?
A) Nothing—rippled has internal time correction
B) Validation messages may be timestamped incorrectly and rejected by other validators
C) The server automatically shuts down
D) Transactions are delayed but eventually process
Correct Answer: B
Explanation: Consensus is time-sensitive. Validation messages include timestamps, and other validators may reject messages that appear to come from the future or distant past. A validator with significant clock skew will have lower agreement percentages because its messages are discarded. This directly impacts reputation and UNL consideration.
4. System Tuning (Tests Technical Understanding):
Why is vm.swappiness = 10 recommended for validator servers?
A) It improves network performance
B) It reduces the likelihood of using swap space, which causes severe performance degradation for memory-intensive applications
C) It increases available RAM
D) It's required by rippled
Correct Answer: B
Explanation: The swappiness parameter controls how aggressively Linux moves memory pages to swap. Default (60) means moderate use of swap. For validators, swap usage causes dramatic performance degradation because rippled keeps critical data structures in memory. Setting swappiness to 10 tells Linux to strongly prefer keeping data in RAM, only using swap under memory pressure. Combined with adequate RAM sizing, this keeps the validator running at full performance.
5. Security Practices (Tests Holistic Understanding):
A server administrator argues that hardening is "overkill for a validator—no one will target it." What's the most accurate response?
A) They're correct—validators are low-value targets
B) Automated scanning and attacks target ALL internet-facing servers regardless of function; hardening prevents opportunistic compromise
C) Hardening is only needed for UNL validators
D) Security is only necessary if storing funds
Correct Answer: B
Explanation: Internet-facing servers are constantly scanned by automated tools looking for weak SSH passwords, unpatched vulnerabilities, and common misconfigurations. These attacks don't know or care that it's a validator—they're looking for any server to compromise for botnets, crypto mining, or other purposes. A compromised validator damages your reputation, potentially the network's perception of you, and could be used to attack other systems. Basic hardening prevents the most common attack vectors at minimal cost.
- Ubuntu Server Guide - Security - https://ubuntu.com/server/docs/security-introduction
- CIS Ubuntu Linux Benchmark - Industry-standard hardening guide
- Lynis Documentation - https://cisofy.com/lynis/
- OpenSSH Best Practices - https://www.ssh.com/academy/ssh/security
- fail2ban Documentation - https://www.fail2ban.org/
- Ubuntu Time Synchronization - https://ubuntu.com/server/docs/network-ntp
- Chrony Documentation - https://chrony.tuxfamily.org/doc/
- Linux Network Tuning - https://wiki.archlinux.org/title/sysctl
- TCP Performance Tuning - Various kernel documentation
For Next Lesson:
With a hardened operating system ready, Lesson 4 will walk through rippled installation using Ubuntu's package manager. Your server should now be ready for the rippled repository configuration and installation.
End of Lesson 3
Total words: ~5,400
Estimated completion time: 60 minutes reading + 2-4 hours for implementation
Key Takeaways
Create a non-root user and disable root SSH login
before any other configuration; running as root is the highest-risk operational pattern.
SSH key-only authentication with fail2ban
eliminates password brute-force attacks, the most common compromise vector for internet-facing servers.
UFW default-deny incoming policy
with explicit allowlist ensures only intended traffic reaches the server; allow SSH and peer port (51235), deny everything else.
NTP time synchronization is mandatory for validators
—clock skew causes validation messages to be rejected, directly impacting your agreement percentage and reputation.
System tuning (sysctl, file descriptors)
optimizes the operating system for rippled's network-intensive operations; defaults are insufficient for production validators. ---