Splunk HEC Secure

PoC to secure HEC using Let's Encrypt certificates.

Generate Certs for Web

Steps

• Create the certificates

splunk-hec-secure/scripts/create-certs.sh

Lines 3 to 31 in 0c991a9

	# Must run as root
	if [ $EUID -ne 0 ]
	then
	echo "Please run this script with root privileges"
	exit 1
	fi
	# Setup context
	APP_DIR=$PWD/configs/mycerts
	DOMAIN=dessy.one
	SPLUNK_HOST=splunk
	FQDN=${SPLUNK_HOST}.${DOMAIN}
	ROOT_CA=isrgrootx1.pem
	# Create cert
	# Use standalone mode as no Web server exists yet.
	certbot certonly --standalone -d $FQDN
	cd /etc/letsencrypt/live/$FQDN
	# Get Let's Encrypt Root CA
	wget -q https://letsencrypt.org/certs/$ROOT_CA -O $APP_DIR/$ROOT_CA
	# Add Certs to the Splunk cert store
	cp fullchain.pem privkey.pem $APP_DIR
	# Create chain of certs for HEC:
	# https://community.splunk.com/t5/All-Apps-and-Add-ons/How-do-I-secure-the-event-collector-port-8088-with-an-ssl/m-p/571431/highlight/true#M75360
	cat cert.pem privkey.pem chain.pem > $APP_DIR/hec.pem

At the end of the script the following should be in $SPLUNK_HOME/etc/auth/mycerts

-rw-r--r--. 1 splunk splunk 5242 Feb 12 16:06 fullchain.pem
-rw-r--r--. 1 splunk splunk 5483 Feb 12 16:06 hec.pem
-rw-r--r--. 1 splunk splunk 1939 Feb 12 16:06 isrgrootx1.pem
-rw-------. 1 splunk splunk  241 Feb 12 16:06 privkey.pem

• Use the following for inputs.conf

[http]
disabled = 0
index = your-hec-index-name
enableSSL = 1
serverCert = /etc/letsencrypt/live/your-server-hostname/hec.pem
sslPassword =
crossOriginSharingPolicy = *

Test HEC

Send a test event:

DOMAIN=dessy.one
SPLUNK_HOST=splunk
FQDN=${SPLUNK_HOST}.${DOMAIN}
curl https://$FQDN:8088/services/collector/event \
-H "Authorization: Splunk abcd-1234-efgh-5678" \
-d '{"event":"hello world"}' -v

Troubleshooting

Check for cert chain integrity:

DOMAIN=dessy.one
SPLUNK_HOST=splunk
FQDN=${SPLUNK_HOST}.${DOMAIN}
openssl s_client -connect $FQDN:8000
openssl s_client -connect $FQDN:8088

Renew certificate

Run the following ideally once per day:

sudo certbot renew