Alertmanager with Slack

https://grafana.com/blog/2020/02/25/step-by-step-guide-to-setting-up-prometheus-alertmanager-with-slack-pagerduty-and-gmail/

https://x.slack.com/services/y?added=1
dev-alerts
Webhook URL
https://hooks.slack.com/services/x/y/z

在k8s上面部署alertmanager deployment
https://devopscube.com/alert-manager-kubernetes-guide/

AlertManager config override yaml
helm show values prometheus-community/prometheus > prometheus-overrides.yaml

1133 alertmanager:
1134   ## If false, alertmanager will not be installed
1135   ##
1136   enabled: true
1137 
1138   config:
1139     route:
1140       group_wait: 10s
1141       group_interval: 10s
1142       repeat_interval: 1h
1143       receiver: 'slack'
1144 
1145     receivers:
1146       - name: 'slack'
1147         slack_configs:
1148         - api_url: https://hooks.slack.com/services/x/y/z
1149           channel: '#dev-alerts'

helm upgrade -i prometheus prometheus-community/prometheus  --values prometheus-overrides.yaml    --namespace prometheus     --set alertmanager.persistentVolume.storageClass="gp2",server.persistentVolume.storageClass="gp2"

更多告警规则
https://samber.github.io/awesome-prometheus-alerts/rules.html

发送的告警内容样式模板

https://github.com/prometheus-community/helm-charts/blob/cc2c507e4a4b625f561577bba13795b7f8bde40a/charts/prometheus/templates/deploy.yaml#L93
  72     extraConfigmapMounts: []
  73       - name: alertmanager-templates
  74         mountPath: /etc/alertmanager
  75         subPath: ""
  76         configMap: alertmanager-templates
  77         readOnly: true

https://devopscube.com/alert-manager-kubernetes-guide/
git clone https://github.com/bibinwilson/kubernetes-alert-manager.git
cat /opt/monitor/prometheus/AlertTemplateConfigMap.yaml

https://hodovi.cc/blog/creating-awesome-alertmanager-templates-for-slack/

cat /opt/monitor/prometheus/AlertTemplateConfigMap.yml