monitor

Check that recent objects exist under configured prefixes for one host.

s3m monitor <host>

monitor reads rules from hosts.<host>.buckets in ~/.config/s3m/config.yml.

Example config

yaml

---
hosts:
  backup:
    region: us-east-2
    access_key: ACCESSKEY
    secret_key: SECRETKEY
    buckets:
      bucket_A:
        - prefix: backups/daily
          suffix: .log
          age: 1d
          size: 30720
      bucket_B:
        - prefix: foo
          age: 12h
          size: 1024
        - prefix: path/to/logs/
          age: 12h

Each rule checks:

prefix: S3 key prefix to scan
suffix: optional client-side suffix filter
age: maximum age for a matching object
size: minimum expected size in bytes

Supported age forms:

integer seconds such as 86400
30s
15m
12h
7d

Output

Prometheus format is the default:

s3m monitor backup

The exported metrics are:

s3m_object_exists
s3m_check_error
s3m_size_mismatch

Each metric includes host, bucket, and prefix labels, plus suffix when configured.

--format

Use InfluxDB line protocol instead:

s3m monitor backup --format influxdb

--exit-on-check-failure

Print metrics first and then exit non-zero if any rule has no recent object, returns a listing error, or only matches objects that are too small:

s3m monitor backup --exit-on-check-failure

Cron

Run s3m monitor from cron when you want periodic checks without a long-running process. The examples below assume the user's own crontab from crontab -e, so they omit the extra user field. They run twice a day at 00:00 and 12:00.

Cron to push Prometheus format to vmagent

txt

0 0,12 * * * s3m monitor backup \
  | curl -sf --max-time 10 \
         -X POST http://localhost:8429/api/v1/import/prometheus \
         --data-binary @-

This keeps the default Prometheus output and pushes it directly to vmagent.

Cron to push InfluxDB format to vmagent

txt

0 0,12 * * * s3m monitor backup --format influxdb \
  | curl -sf --max-time 10 \
         -X POST http://localhost:8429/influx/write \
         --data-binary @-

Use this if your vmagent setup already expects InfluxDB line protocol on /influx/write.

Cron to push with an extra host label

txt

0 0,12 * * * s3m monitor backup --format influxdb \
  | curl -sf --max-time 10 \
         -X POST "http://localhost:8429/influx/write?extra_label=host=$(hostname -s)" \
         --data-binary @-

That adds a host label on every series received by vmagent.

Cron wrapper when check failures must also fail the job

If you use --exit-on-check-failure, a plain pipeline usually returns the exit status from curl, not from s3m monitor. Use a wrapper script instead. This version is copy-pasteable and sends an HTML email if either the push fails or the monitor checks fail:

#!/usr/bin/env bash
set -u

ERROR_EMAIL="[email protected]"
MONITOR_HOST="backup"
VMAGENT_URL="http://localhost:8429/api/v1/import/prometheus"
SENDMAIL_BIN="/usr/sbin/sendmail"
LOCAL_HOSTNAME="$(hostname -f 2>/dev/null || hostname)"

metrics_file="$(mktemp)"
monitor_stderr="$(mktemp)"
curl_stderr="$(mktemp)"

cleanup() {
  rm -f "$metrics_file" "$monitor_stderr" "$curl_stderr"
}
trap cleanup EXIT

html_escape() {
  sed \
    -e 's/&/\&amp;/g' \
    -e 's/</\&lt;/g' \
    -e 's/>/\&gt;/g'
}

monitor_status=0
curl_status=0

s3m monitor "$MONITOR_HOST" --exit-on-check-failure \
  >"$metrics_file" \
  2>"$monitor_stderr" || monitor_status=$?

if [ -s "$metrics_file" ]; then
  curl -sf --max-time 10 \
    -X POST "$VMAGENT_URL" \
    --data-binary @"$metrics_file" \
    2>"$curl_stderr" || curl_status=$?
fi

if [ "$monitor_status" -ne 0 ] || [ "$curl_status" -ne 0 ]; then
  subject="ALERT: s3m monitor failure on $LOCAL_HOSTNAME"
  email_headers=$(
    cat <<EOF
To: $ERROR_EMAIL
Subject: $subject
Mime-Version: 1.0
Content-Type: text/html; charset=utf-8

<html><head><style>
body { font-family: sans-serif; }
pre { font-family: monospace; white-space: pre-wrap; margin: 0; }
</style></head><body>
EOF
  )

  {
    printf '%s\n' "$email_headers"
    printf '<h2>%s</h2>\n' "$subject"
    printf '<p><strong>Host:</strong> %s</p>\n' "$LOCAL_HOSTNAME"
    printf '<p><strong>Monitor host:</strong> %s</p>\n' "$MONITOR_HOST"
    printf '<p><strong>s3m monitor exit code:</strong> %s<br>\n' "$monitor_status"
    printf '<strong>curl exit code:</strong> %s</p>\n' "$curl_status"

    printf '<h3>s3m monitor stderr</h3><pre>'
    if [ -s "$monitor_stderr" ]; then
      html_escape <"$monitor_stderr"
    else
      printf 'no stderr output'
    fi
    printf '</pre>\n'

    printf '<h3>curl stderr</h3><pre>'
    if [ -s "$curl_stderr" ]; then
      html_escape <"$curl_stderr"
    else
      printf 'no stderr output'
    fi
    printf '</pre>\n'

    printf '<h3>metrics payload</h3><pre>'
    if [ -s "$metrics_file" ]; then
      html_escape <"$metrics_file"
    else
      printf 'no metrics were produced'
    fi
    printf '</pre>\n'

    printf '</body></html>\n'
  } | "$SENDMAIL_BIN" -t

  exit 1
fi

Example cron entry:

txt

0 0,12 * * * /usr/local/bin/s3m-monitor-vmagent-alert.sh

The script exits with status 1 after sending the email.

monitor ​

Example config ​

Output ​

--format ​

--exit-on-check-failure ​

Cron ​

Cron to push Prometheus format to vmagent ​

Cron to push InfluxDB format to vmagent ​

Cron to push with an extra host label ​

Cron wrapper when check failures must also fail the job ​