Alertmanager 는 Prometheus 가 보낸 경보를 받아 묶고(grouping), 중복을 없애고(deduplication), 종료한 경보를 누르고(inhibition), 메일 · Slack · webhook 으로 보낸다. 규칙 자체는 Prometheus 가 평가하므로 Alertmanager 에는 규칙을 두지 않는다.
현행 버전은 0.34.1[1] 이다. Prometheus 3.x 는 Alertmanager v1 API 를 더 이상 쓰지 않으므로 0.16.0 이상이어야 한다.
AM_VER=0.34.1
wget https://github.com/prometheus/alertmanager/releases/download/v${AM_VER}/alertmanager-${AM_VER}.linux-amd64.tar.gz
tar -xvzf alertmanager-${AM_VER}.linux-amd64.tar.gz
sudo cp alertmanager-${AM_VER}.linux-amd64/alertmanager /usr/local/bin/
sudo mkdir -p /etc/alertmanager /var/log/alertmanager/data
기본 포트는 9093(웹 · API), 9094(클러스터 gossip) 다.
/etc/systemd/system/alertmanager.service
[Unit]
Description=Prometheus Alertmanager
Wants=network-online.target
After=network-online.target
[Service]
User=alertmanager
Group=alertmanager
Type=simple
Environment=CONFIG_FILE="/etc/alertmanager/alertmanager.yml"
Environment=STORAGE_PATH="/var/log/alertmanager/data"
Environment=WEB_LISTEN_ADDRESS="0.0.0.0:9093"
Environment=CLUSTER_LISTEN_ADDRESS="0.0.0.0:9094"
ExecStart=/usr/local/bin/alertmanager \
--config.file=${CONFIG_FILE} \
--storage.path=${STORAGE_PATH} \
--web.listen-address=${WEB_LISTEN_ADDRESS} \
--cluster.listen-address=${CLUSTER_LISTEN_ADDRESS}
Restart=on-failure
[Install]
WantedBy=multi-user.target
경보 규칙은 Prometheus 가 rule_files 로 읽어 평가하고, 조건이 맞으면 Alertmanager 로 보낸다. Alertmanager 설정에는 규칙을 두지 않고 경로(route) 와 수신자(receiver) 만 둔다. 규칙 예시는 Prometheus 문서의 ruls_alert_cpu.yml 을 본다.
/etc/alertmanager/alertmanager.yml
route:
group_by: ['alertname', 'instance']
group_wait: 30s
group_interval: 5m
repeat_interval: 4h
receiver: webhook
receivers:
- name: webhook
webhook_configs:
- url: http://localhost:5000/alert
Prometheus 쪽에서 Alertmanager 를 가리킨다.
alerting:
alertmanagers:
- static_configs:
- targets: ['localhost:9093']
curl -s http://localhost:9093/-/healthy
amtool --alertmanager.url=http://localhost:9093 config show
from flask import Flask, request, jsonify
import requests
app = Flask(__name__)
API_URL = 'https://openapi.haedong.net/mail/api/v2.0/mails/send?userID=sksksk'
CID = 'cidcidcid'
TOKEN = '${MASKED}'
SENDER = 'haedong@haedong.net'
@app.route('/alert', methods=['POST'])
def alert():
data = request.json
for alert in data.get("alerts", []):
subject = alert["annotations"].get("summary", "No summary")
body = alert["annotations"].get("description", "No description")
payload = {
"sender": SENDER,
"subject": subject,
"body": body
}
headers = {
"cid": CID,
"token": TOKEN,
"Content-Type": "application/json"
}
response = requests.post(API_URL, headers=headers, json=payload)
print(f"Sent alert: {response.status_code} - {response.text}")
return "OK", 200
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000)
#!/usr/bin/env python3
import os
import sys
import json
import subprocess
import logging
from flask import Flask, request, abort
SCRIPT = "/tmp/alert.sh"
LOG_FILE = "/var/log/am_webhook.log"
RAW_ALERT_FILE = "/tmp/alert_raw.json"
# === Logging to stdout and file ===
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s %(levelname)s: %(message)s',
handlers=[
logging.FileHandler(LOG_FILE),
logging.StreamHandler(sys.stdout) # print to terminal
]
)
app = Flask(__name__)
@app.route('/alert', methods=['POST'])
def alert():
if not request.is_json:
abort(400, "Expected JSON")
data = request.get_json()
# Save full JSON payload
try:
with open(RAW_ALERT_FILE, 'a') as f:
json.dump(data, f, indent=2)
f.write("\n\n")
except Exception as e:
logging.error(f"Failed to write raw alert JSON: {e}")
# Process and trigger script
alerts = data.get("alerts", [])
for alert in alerts:
status = alert.get("status", "")
alertname = alert.get("labels", {}).get("alertname", "UnknownAlert")
if status == "firing":
logging.info(f"Running script for alert: {alertname}")
try:
subprocess.Popen([SCRIPT, alertname])
except Exception as e:
logging.error(f"Failed to run script: {e}")
else:
logging.info(f"Ignored alert: {alertname} (status: {status})")
return "OK", 200
if __name__ == '__main__':
app.run(host="0.0.0.0", port=5001)
최신 버전 0.34.1 (2026-09-17) — 2026-09-20 확인. https://prometheus.io/download/ ↩︎