> ## Documentation Index
> Fetch the complete documentation index at: https://docs.apinizer.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Prometheus & Grafana

> Prometheus ve Grafana kurulumunu gerçekleştirerek Kubernetes cluster izleme ve görselleştirme yapabilir. Prometheus ile metrik verilerini toplayabilir, Grafana ile dashboard'lar oluşturabilir ve kube-state-metrics ile cluster kaynak durumlarını izleyebilir.

## PROMETHEUS

Prometheus, Kubernetes ve mikro hizmet mimarileri için tasarlanmış açık kaynak sistem ve ağ izleme aracıdır. Performans verilerini toplar, depolar, sorgular ve grafikler oluşturur.

## 1) Prometheus Kurulumu

Çalışılacak olan ortam (namespace) oluşturulur:

```bash theme={null}
kubectl create namespace monitoring
```

Rol Tabanlı Erişim Kontrolü (RBAC) oluşturulur:

```bash theme={null}
vi clusterRole.yaml
```

```yaml theme={null}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus
rules:
  - apiGroups: [""]
    resources:
      - nodes
      - nodes/proxy
      - services
      - endpoints
      - pods
    verbs: ["get", "list", "watch"]
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs: ["get", "list", "watch"]
  - nonResourceURLs: ["/metrics"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: prometheus
subjects:
  - kind: ServiceAccount
    name: default
    namespace: monitoring
```

```bash theme={null}
kubectl create -f clusterRole.yaml
```

Kubernetes için Prometheus'un yapılandırma ayarlarını içeren bir ConfigMap tanımlanır:

```bash theme={null}
vi config-map.yaml
```

```yaml theme={null}
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-server-conf
  labels:
    name: prometheus-server-conf
  namespace: monitoring
data:
  prometheus.rules: |-
    groups:
      - name: demo alert rules
        rules:
          - alert: High Pod Memory
            expr: sum(container_memory_usage_bytes) > 1
            for: 1m
            labels:
              severity: slack
            annotations:
              summary: High Memory Usage
  prometheus.yml: |-
    global:
      scrape_interval: 5s
      evaluation_interval: 5s
    rule_files:
      - /etc/prometheus/prometheus.rules
    alerting:
      alertmanagers:
        - scheme: http
          static_configs:
            - targets:
                - "alertmanager.monitoring.svc:9093"
    scrape_configs:
      - job_name: 'node-exporter'
        kubernetes_sd_configs:
          - role: endpoints
        relabel_configs:
          - source_labels: [__meta_kubernetes_endpoints_name]
            regex: 'node-exporter'
            action: keep
      - job_name: 'kubernetes-apiservers'
        kubernetes_sd_configs:
          - role: endpoints
        scheme: https
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        relabel_configs:
          - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
            action: keep
            regex: default;kubernetes;https
      - job_name: 'kubernetes-nodes'
        scheme: https
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        kubernetes_sd_configs:
          - role: node
        relabel_configs:
          - action: labelmap
            regex: __meta_kubernetes_node_label_(.+)
          - target_label: __address__
            replacement: kubernetes.default.svc:443
          - source_labels: [__meta_kubernetes_node_name]
            regex: (.+)
            target_label: __metrics_path__
            replacement: /api/v1/nodes/${1}/proxy/metrics
      - job_name: 'kubernetes-pods'
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
            action: keep
            regex: true
          - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
            action: replace
            target_label: __metrics_path__
            regex: (.+)
          - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
            action: replace
            regex: ([^:]+)(?::\d+)?;(\d+)
            replacement: $1:$2
            target_label: __address__
          - action: labelmap
            regex: __meta_kubernetes_pod_label_(.+)
          - source_labels: [__meta_kubernetes_namespace]
            action: replace
            target_label: kubernetes_namespace
          - source_labels: [__meta_kubernetes_pod_name]
            action: replace
            target_label: kubernetes_pod_name
      - job_name: 'kube-state-metrics'
        static_configs:
          - targets: ['kube-state-metrics.kube-system.svc.cluster.local:8080']
      - job_name: 'kubernetes-cadvisor'
        scheme: https
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        kubernetes_sd_configs:
          - role: node
        relabel_configs:
          - action: labelmap
            regex: __meta_kubernetes_node_label_(.+)
          - target_label: __address__
            replacement: kubernetes.default.svc:443
          - source_labels: [__meta_kubernetes_node_name]
            regex: (.+)
            target_label: __metrics_path__
            replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor
      - job_name: 'kubernetes-service-endpoints'
        kubernetes_sd_configs:
          - role: endpoints
        relabel_configs:
          - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
            action: keep
            regex: true
          - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
            action: replace
            target_label: __scheme__
            regex: (https?)
          - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
            action: replace
            target_label: __metrics_path__
            regex: (.+)
          - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
            action: replace
            target_label: __address__
            regex: ([^:]+)(?::\d+)?;(\d+)
            replacement: $1:$2
          - action: labelmap
            regex: __meta_kubernetes_service_label_(.+)
          - source_labels: [__meta_kubernetes_namespace]
            action: replace
            target_label: kubernetes_namespace
          - source_labels: [__meta_kubernetes_service_name]
            action: replace
            target_label: kubernetes_name
```

```bash theme={null}
kubectl apply -f config-map.yaml
```

Prometheus deployment ve servisi oluşturulur:

```bash theme={null}
vi prometheus-deploy.yaml
```

```yaml theme={null}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-deployment
  namespace: monitoring
  labels:
    app: prometheus-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus-server
  template:
    metadata:
      labels:
        app: prometheus-server
    spec:
      containers:
        - name: prometheus
          image: prom/prometheus
          args:
            - "--storage.tsdb.retention.time=12h"
            - "--config.file=/etc/prometheus/prometheus.yml"
            - "--storage.tsdb.path=/prometheus/"
          ports:
            - containerPort: 9090
          resources:
            requests:
              cpu: 500m
              memory: 500M
            limits:
              cpu: 1
              memory: 1Gi
          volumeMounts:
            - name: prometheus-config-volume
              mountPath: /etc/prometheus/
            - name: prometheus-storage-volume
              mountPath: /prometheus/
      volumes:
        - name: prometheus-config-volume
          configMap:
            defaultMode: 420
            name: prometheus-server-conf
        - name: prometheus-storage-volume
          emptyDir: {}
```

```bash theme={null}
vi prometheus-service.yaml
```

```yaml theme={null}
apiVersion: v1
kind: Service
metadata:
  name: prometheus-service
  namespace: monitoring
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/port: '9090'
spec:
  selector:
    app: prometheus-server
  type: NodePort
  ports:
    - port: 8080
      targetPort: 9090
      nodePort: 30686
```

```bash theme={null}
kubectl apply -f prometheus-deploy.yaml
kubectl apply -f prometheus-service.yaml
```

Oluşturduktan sonra `http://<KUBERNETES_WORKER_IP_ADDRESS>:<NODE_PORT>` adresinden Prometheus'a erişim sağlanır.

## 2) KUBE-STATE-METRICS Kurulumu

Kube-state-metrics, Kubernetes kümesindeki kaynakların (örneğin, pod'lar, servisler, replicaset'ler) durumu hakkında bilgi toplamak için kullanılır.

```bash theme={null}
vi cluster-role-binding.yaml
```

```yaml theme={null}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.3.0
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
  - kind: ServiceAccount
    name: kube-state-metrics
    namespace: kube-system
```

```bash theme={null}
vi cluster-role.yaml
```

```yaml theme={null}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.3.0
  name: kube-state-metrics
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs:
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - statefulsets
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - list
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - list
  - watch
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - list
  - watch
- apiGroups:
  - certificates.k8s.io
  resources:
  - certificatesigningrequests
  verbs:
  - list
  - watch
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  - volumeattachments
  verbs:
  - list
  - watch
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  - validatingwebhookconfigurations
  verbs:
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - networkpolicies
  - ingresses
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
```

```bash theme={null}
kubectl apply -f cluster-role-binding.yaml
kubectl apply -f cluster-role.yaml
```

```bash theme={null}
vi metrics-deployment.yaml
```

```yaml theme={null}
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.3.0
  name: kube-state-metrics
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: kube-state-metrics
  template:
    metadata:
      labels:
        app.kubernetes.io/component: exporter
        app.kubernetes.io/name: kube-state-metrics
        app.kubernetes.io/version: 2.3.0
    spec:
      automountServiceAccountToken: true
      containers:
        - image: k8s.gcr.io/kube-state-metrics/kube-state-metrics:v2.3.0
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 5
          name: kube-state-metrics
          ports:
            - containerPort: 8080
              name: http-metrics
            - containerPort: 8081
              name: telemetry
          readinessProbe:
            httpGet:
              path: /
              port: 8081
            initialDelaySeconds: 5
            timeoutSeconds: 5
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 65534
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: kube-state-metrics
```

```bash theme={null}
kubectl apply -f metrics-deployment.yaml
```

```bash theme={null}
vi service-account.yaml
```

```yaml theme={null}
apiVersion: v1
automountServiceAccountToken: false
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.3.0
  name: kube-state-metrics
  namespace: kube-system
```

```bash theme={null}
vi kube-state-metric-service.yaml
```

```yaml theme={null}
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.3.0
  name: kube-state-metrics
  namespace: kube-system
spec:
  clusterIP: None
  ports:
    - name: http-metrics
      port: 8080
      targetPort: http-metrics
    - name: telemetry
      port: 8081
      targetPort: telemetry
  selector:
    app.kubernetes.io/name: kube-state-metrics
```

```bash theme={null}
kubectl apply -f service-account.yaml
kubectl apply -f kube-state-metric-service.yaml
```

**Detaylı bilgi** için Prometheus'un kendi döküman sayfasını inceleyebilirsiniz: [https://prometheus.io/docs/introduction/overview/](https://prometheus.io/docs/introduction/overview/)

## 3) Grafana Kurulumu

Grafana, açık kaynak kodlu bir veri görselleştirme web uygulamasıdır. Çeşitli veri kaynaklarına bağlayarak verilerinizi çizelgelerde ve grafiklerde görüntüleyebilir ve daha kolay yorumlayabilirsiniz.

<Note>
  Aşağıda Ubuntu işletim sistemleri üzerine Grafana kurulumu anlatılmıştır. Kubernetes üzerinde Grafana kurulumu için deployment yapılandırması kullanılabilir.
</Note>

```bash theme={null}
sudo wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt update
sudo apt-get install -y grafana
sudo systemctl start grafana-server
sudo systemctl status grafana-server
sudo systemctl enable grafana-server
```

Kurulum tamamlandıktan sonra `http://<SERVER_IP_ADDRESS>:3000` adresinden erişim sağlanır.

<Info>
  Grafana için varsayılan giriş bilgileri:

  * **username:** admin
  * **password:** admin
</Info>

## 4) Grafana'da Prometheus Bağlantısı Eklenmesi

Sol menüden bir tane Prometheus connection bilgisi eklenir.

Prometheus tercih edilir. "Add new data source" seçeneği tıklanarak, Prometheus server URL bilgisi girilir ve bağlantı kaydedilir.

<Frame caption="Grafana - Prometheus veri kaynağı ekleme">
  <img src="https://mintcdn.com/apinizer/rfZaKpIysRC7U9dY/images/setup/kubernetes/prometheus-grafana/01-prometheus-data-source.png?fit=max&auto=format&n=rfZaKpIysRC7U9dY&q=85&s=40c1d301ed20e5565338e23a2f18c808" alt="Prometheus Veri Kaynağı Ekleme" width="975" height="623" style={{ borderRadius: '0.5rem' }} data-path="images/setup/kubernetes/prometheus-grafana/01-prometheus-data-source.png" />
</Frame>

Sol menüden "Dashboard" sekmesine tıklanır. Ardından, "New" seçeneği ile yeni bir import işlemi başlatılır.

<Frame caption="Grafana - Dashboard import ekranı">
  <img src="https://mintcdn.com/apinizer/rfZaKpIysRC7U9dY/images/setup/kubernetes/prometheus-grafana/02-dashboard-import.png?fit=max&auto=format&n=rfZaKpIysRC7U9dY&q=85&s=30e9fd97b3f633d60d1cd70de86a4f24" alt="Dashboard Import Ekranı" width="972" height="247" style={{ borderRadius: '0.5rem' }} data-path="images/setup/kubernetes/prometheus-grafana/02-dashboard-import.png" />
</Frame>

Sırasıyla ID alanına **315** ve **13498** numaraları eklenir. Load butonuna tıklanır. Dashboard'ın çalışacağı Prometheus bağlantısı seçilir ve "Import" seçeneği ile kaydedilir.

<CardGroup cols={2}>
  <Card title="Dashboard ID: 315" icon="chart-bar">
    Cluster ve node'larınızdaki CPU kullanımı, bellek kullanımı ve ağ üzerinde gerçekleşen işlem yoğunluğu gibi bilgileri görsel olarak sunar.
  </Card>

  <Card title="Dashboard ID: 13498" icon="chart-line">
    Namespace, pod ve container düzeyindeki CPU kullanımı, bellek kullanımı ve ağ üzerindeki işlem yoğunluğu bilgilerini görsel olarak sunar.
  </Card>
</CardGroup>

**Detaylı bilgi** için Grafana'nın kendi döküman sayfasını inceleyebilirsiniz: [https://grafana.com/docs/grafana/latest/](https://grafana.com/docs/grafana/latest/)