[{"content":"CloudflareとGithubの連携 Cloudflareのダッシュボードページを開きます\n右上の\u0026quot;追加\u0026quot;を押し、\u0026ldquo;Pages\u0026quot;を押します ページが変わり、\u0026ldquo;既存のGitリポジトリをインポートする\u0026quot;を押します アカウントからサイトをデプロイするという画面になったら、下の方にある\u0026quot;Githubに接続\u0026quot;を押します Githubの認証ページが表示されるので許可します\nこのとき\u0026quot;All repositories\u0026quot;でも良いが、必要なリポジトリのみ許可したいので\u0026quot;Only select repositories\u0026quot;を選択する\nその後、許可するリポジトリを選択します\n下の方にある\u0026quot;Install \u0026amp; Authorize\u0026quot;を押す 無事認証が通ったら\u0026quot;セットアップの開始\u0026quot;を押します ビルドとデプロイ \u0026ldquo;ビルドとデプロイのセットアップ\u0026quot;の画面になります\n\u0026ldquo;プロジェクト名\u0026quot;と\u0026quot;プロダクションブランチ\u0026quot;を入力・選択します\n下の方のビルドの設定は、今回はHTML等のファイルを直接リポジトリにプッシュしているので特に設定しません\n※HTML等のファイルはAIを用いて作成しました\n\u0026ldquo;保存してデプロイする\u0026quot;を押します \u0026ldquo;ビルドおよびデプロイを実行しています\u0026quot;と表示されるので待ちます 完了すると\u0026quot;成功しました！\u0026hellip;\u0026ldquo;と表示されます 表示されているURLに行くとデプロイしたサイトが表示されています ","date":"2026-03-20T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/cloudflare-pages/","title":"Cloudflare Pagesを使ってWebページを公開する"},{"content":"環境 Ubuntu 24.04.1 LTS Docker 29.1.3 Traefik 3.6.5 Harbor v2.11.1 上記構成の理由はSSL証明書を自分で管理するのではなく自動更新かつワイルドカードでの取得にしたかったため\nIPの固定 IPアドレスを固定します\n1 2 ip a nano 99-config.yaml 99-config.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 network: version: 2 renderer: networkd ethernets: ens18: dhcp4: false addresses: - 192.168.10.70/24 routes: - to: default via: 192.168.10.1 nameservers: search: [] addresses: [192.168.10.1] 設定を適用する\n1 2 3 sudo cp 99-config.yaml /etc/netplan/ sudo netplan apply sudo chmod 600 /etc/netplan/99-config.yaml 設定ファイルの書き換え1（Harbor） Harborの設定ファイルを書き換える\n1 nano harbor.yml 下記の設定を変更する\n\u0026ldquo;hostname\u0026rdquo;,\u0026ldquo;harbor.internal.example.dev\u0026rdquo; \u0026ldquo;http\u0026rdquo;,\u0026ldquo;port:8088\u0026rdquo; ポートがTraefikの受付ポート等とかぶらないようにするため \u0026ldquo;http\u0026rdquo;,\u0026ldquo;relativeurls: true\u0026rdquo; デフォルトではない項目なので項目を追加する \u0026ldquo;external_url: https://harbor.internal.example.dev\u0026rdquo; \u0026ldquo;http\u0026quot;ではなく\u0026quot;https\u0026ldquo;のURLを記載する 下記コマンドを実行し設定ファイル作成（再作成）する\n1 sudo ./prepare 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 mao@harbor-server:~/harbor$ sudo ./prepare prepare base dir is set to /home/mao/harbor WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https Clearing the configuration file: /config/portal/nginx.conf Clearing the configuration file: /config/db/env Clearing the configuration file: /config/jobservice/config.yml Clearing the configuration file: /config/jobservice/env Clearing the configuration file: /config/log/rsyslog_docker.conf Clearing the configuration file: /config/log/logrotate.conf Clearing the configuration file: /config/registry/root.crt Clearing the configuration file: /config/registry/config.yml Clearing the configuration file: /config/registry/passwd Clearing the configuration file: /config/core/env Clearing the configuration file: /config/core/app.conf Clearing the configuration file: /config/registryctl/config.yml Clearing the configuration file: /config/registryctl/env Clearing the configuration file: /config/nginx/nginx.conf Generated configuration file: /config/portal/nginx.conf Generated configuration file: /config/log/logrotate.conf Generated configuration file: /config/log/rsyslog_docker.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/registryctl/config.yml Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml loaded secret from file: /data/secret/keys/secretkey Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir mao@harbor-server:~/harbor$ 設定ファイルの書き換え2（docker-compose） 上記で作成されたcomposeファイルを編集する\n1 nano docker-compose.yml 下記項目について変更する\nnginx labels networks Traefikの設定例 Traefikでhttpsにするには下記のように\u0026quot;labels\u0026quot;と\u0026quot;networks\u0026quot;を設定する\n例：compose.nginx.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 services: nginx: image: nginx:1.29.4 container_name: nginx-sample restart: unless-stopped networks: - traefik-public labels: # Traefik で公開する - \u0026#34;traefik.enable=true\u0026#34; # ルーター設定（実際のドメインに変更してください） - \u0026#34;traefik.http.routers.nginx.rule=Host(`nginx.example.dev`)\u0026#34; - \u0026#34;traefik.http.routers.nginx.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.nginx.tls.certresolver=letsencrypt\u0026#34; # ワイルドカード証明書の設定 - \u0026#34;traefik.http.routers.nginx.tls.domains[0].main=example.dev\u0026#34; - \u0026#34;traefik.http.routers.nginx.tls.domains[0].sans=*.example.dev\u0026#34; # サービス設定 - \u0026#34;traefik.http.services.nginx.loadbalancer.server.port=80\u0026#34; networks: traefik-public: name: traefik-public external: true composeファイルを書き換える docker-compose.yml 上記ファイルの下の方にnginx(proxy)の設定があります 下記は抜粋です 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 proxy: image: goharbor/nginx-photon:v2.11.1 container_name: nginx restart: always cap_drop: - ALL cap_add: - CHOWN - SETGID - SETUID - NET_BIND_SERVICE volumes: - ./common/config/nginx:/etc/nginx:z - /data/secret/cert:/etc/cert:z - type: bind source: ./common/config/shared/trust-certificates target: /harbor_cust_cert networks: - harbor + # Traefik用追加設定 + - traefik-public - #ports: - # - 8088:8080 + # 追加（外部には公開しない） + expose: + - \u0026#34;8080\u0026#34; depends_on: - registry - core - portal - log logging: driver: \u0026#34;syslog\u0026#34; options: syslog-address: \u0026#34;tcp://localhost:1514\u0026#34; tag: \u0026#34;proxy\u0026#34; + # Traefik用追加設定 + labels: + # Traefik で公開する + - \u0026#34;traefik.enable=true\u0026#34; + # ルーター設定（実際のドメインに変更してください） + - \u0026#34;traefik.http.routers.harbor.rule=Host(`harbor.internal.example.dev`)\u0026#34; + - \u0026#34;traefik.http.routers.harbor.entrypoints=websecure\u0026#34; + - \u0026#34;traefik.http.routers.harbor.tls.certresolver=letsencrypt\u0026#34; + # ワイルドカード証明書の設定 + - \u0026#34;traefik.http.routers.harbor.tls.domains[0].main=internal.example.dev\u0026#34; + - \u0026#34;traefik.http.routers.harbor.tls.domains[0].sans=*internal.example.dev\u0026#34; + # サービス設定 + - \u0026#34;traefik.http.services.harbor.loadbalancer.server.port=8080\u0026#34; networks: harbor: external: false + traefik-public: + name: traefik-public + external: true 起動 順番としてはTraefikが起動してからHarborを起動する\n下記コマンドで起動する\n1 sudo docker compose -up -d しばらくすると起動するのでダッシュボードにアクセスできるようになる\n\u0026ldquo;gateway timeout\u0026quot;が表示される場合は一度落としてから再度起動する\nこれでhttpsでアクセス可能かつTraefikでSSL証明書を管理できるようになりました\n参考URL https://github.com/goharbor/harbor/issues/12135 https://medium.com/@joelkoussawo/installing-harbor-container-registry-behind-traefik-reverse-proxy-with-lets-encrypt-certificate-8ab1733daa1 https://tech-mmmm.blogspot.com/2022/12/ossharbor.html ","date":"2026-03-08T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/traefik-harbor-ssl/","title":"Traefikを使ってHarborをHTTPSで通信させる"},{"content":"VM準備 \u0026ldquo;qemu-guest-agent\u0026quot;と\u0026quot;nano\u0026quot;をインストールする\n1 sudo apt install qemu-guest-agent nano \u0026ldquo;Docker\u0026quot;をインストールする\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Add Docker\u0026#39;s official GPG key: sudo apt update sudo apt install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources: sudo tee /etc/apt/sources.list.d/docker.sources \u0026lt;\u0026lt;EOF Types: deb URIs: https://download.docker.com/linux/ubuntu Suites: $(. /etc/os-release \u0026amp;\u0026amp; echo \u0026#34;${UBUNTU_CODENAME:-$VERSION_CODENAME}\u0026#34;) Components: stable Signed-By: /etc/apt/keyrings/docker.asc EOF sudo apt update 1 sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin バージョン確認\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 mao@traefik-server:~$ sudo docker version Client: Docker Engine - Community Version: 29.1.3 API version: 1.52 Go version: go1.25.5 Git commit: f52814d Built: Fri Dec 12 14:49:32 2025 OS/Arch: linux/amd64 Context: default Server: Docker Engine - Community Engine: Version: 29.1.3 API version: 1.52 (minimum version 1.44) Go version: go1.25.5 Git commit: fbf3ed2 Built: Fri Dec 12 14:49:32 2025 OS/Arch: linux/amd64 Experimental: false containerd: Version: v2.2.1 GitCommit: dea7da592f5d1d2b7755e3a161be07f43fad8f75 runc: Version: 1.3.4 GitCommit: v1.3.4-0-gd6d73eb8 docker-init: Version: 0.19.0 GitCommit: de40ad0 Traefikの設定 設定ファイルの一部をAI(Claude Opus 4.5,Gemini 3 Flash)に生成してもらっています\nファイル構成 フォルダ構成は下記の通り\n1 2 3 4 5 6 7 traefik-docker/ ├── .env # 環境変数 ├── compose-nginx.yaml # nginxサンプルアプリ設定（個別） ├── compose.yaml # Traefik本体の設定 ├── dynamic.yaml # Traefikの動的設定 └── traefik.yaml # Traefikの静的設定 └── acme.json # 取得したSSL証明書が保存されるファイル ファイルの中身 compose.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 services: traefik: image: traefik:3.6.5 container_name: traefik restart: unless-stopped ports: # HTTP - \u0026#34;80:80\u0026#34; # HTTPS - \u0026#34;443:443\u0026#34; # ダッシュボード（開発環境のみ） - \u0026#34;8080:8080\u0026#34; volumes: # 静的設定ファイル - ./traefik.yaml:/etc/traefik/traefik.yaml:ro # 動的設定ファイル - ./dynamic.yaml:/etc/traefik/dynamic.yaml:ro # Let\u0026#39;s Encrypt 証明書保存先 - ./acme.json:/etc/traefik/acme.json # Docker ソケット（Docker プロバイダー使用時） - /var/run/docker.sock:/var/run/docker.sock:ro # カスタム証明書（任意） # - ./certs:/etc/traefik/certs:ro networks: - traefik-public # 環境変数設定（Cloudflare DNS チャレンジ用） # .env ファイルから読み込み、明示的に設定 env_file: - .env environment: CF_DNS_API_TOKEN: ${CF_DNS_API_TOKEN} TZ: \u0026#34;Asia/Tokyo\u0026#34; labels: - traefik.enable=true - traefik.docker.network=traefik-public - traefik.http.services.traefik-dashboard.loadbalancer.server.port=8080 - traefik.http.routers.traefik-dashboard-http.entrypoints=http #- traefik.http.routers.traefik-dashboard-http.rule=Host(`traefik.local`) command: - \u0026#34;--api.dashboard=true\u0026#34; - \u0026#34;--api.insecure=true\u0026#34; - \u0026#34;--providers.docker=true\u0026#34; - \u0026#34;--entrypoints.http.address=:80\u0026#34; - \u0026#34;--providers.docker.exposedByDefault=false\u0026#34; networks: traefik-public: name: traefik-public .env 1 2 3 4 # ============================================================ # Cloudflare 認証設定 # ============================================================ CF_DNS_API_TOKEN=***** traefik.yaml\nSSL証明書をステージング用と本番用を切り替えられるように記載している 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 # ============================================================ # Traefik 静的設定ファイル (traefik.yml) # ============================================================ # このファイルはTraefikの起動時に読み込まれる静的設定です。 # 変更を反映するにはTraefikの再起動が必要です。 # ------------------------------------------------------------ # グローバル設定 # ------------------------------------------------------------ global: # 匿名の使用統計を送信するかどうか checkNewVersion: true # 新バージョンのチェックを行うかどうか sendAnonymousUsage: false # ------------------------------------------------------------ # API / ダッシュボード設定 # ------------------------------------------------------------ api: # ダッシュボードを有効にする dashboard: true # APIを有効にする（本番環境では false 推奨） insecure: true # デバッグモードを有効にする debug: false # ------------------------------------------------------------ # ログ設定 # ------------------------------------------------------------ log: # ログレベル: DEBUG, INFO, WARN, ERROR, FATAL, PANIC level: INFO # ログ出力先（省略時は標準出力） # filePath: \u0026#34;/var/log/traefik/traefik.log\u0026#34; # ログフォーマット: common, json format: common # ------------------------------------------------------------ # アクセスログ設定 # ------------------------------------------------------------ accessLog: # アクセスログ出力先（省略時は標準出力） # filePath: \u0026#34;/var/log/traefik/access.log\u0026#34; # ログフォーマット: common, json format: common # バッファリングサイズ（パフォーマンス向上のため） bufferingSize: 100 # フィルタリング設定 filters: # ステータスコードでフィルタ statusCodes: - \u0026#34;200-299\u0026#34; - \u0026#34;400-499\u0026#34; - \u0026#34;500-599\u0026#34; # 最小リクエスト時間でフィルタ（ms） # minDuration: \u0026#34;10ms\u0026#34; # ------------------------------------------------------------ # エントリーポイント設定 # ------------------------------------------------------------ # エントリーポイントは外部からのトラフィックを受け付けるポートです entryPoints: # HTTP エントリーポイント web: address: \u0026#34;:80\u0026#34; # HTTPからHTTPSへのリダイレクト設定 http: redirections: entryPoint: to: websecure scheme: https permanent: true # HTTPS エントリーポイント websecure: address: \u0026#34;:443\u0026#34; http: tls: # デフォルトの証明書リゾルバー certResolver: letsencrypt # TLSオプション（任意） # options: default # カスタムポート例（API用など） # api: # address: \u0026#34;:8080\u0026#34; # ------------------------------------------------------------ # 証明書リゾルバー設定 (Let\u0026#39;s Encrypt / ACME + Cloudflare) # ------------------------------------------------------------ certificatesResolvers: # Cloudflare DNS-01 チャレンジ（ワイルドカード証明書対応） letsencrypt: acme: # Let\u0026#39;s Encrypt アカウント用メールアドレス #email: \u0026#34;your-email@example.com\u0026#34; email: \u0026#34;\u0026#34; # 証明書の保存先 storage: \u0026#34;/etc/traefik/acme.json\u0026#34; # 本番用エンドポイント（テスト完了後にこちらを使用） # caServer: \u0026#34;https://acme-v02.api.letsencrypt.org/directory\u0026#34; # テスト用エンドポイント（レート制限回避、最初はこちらで動作確認） caServer: \u0026#34;https://acme-staging-v02.api.letsencrypt.org/directory\u0026#34; # Cloudflare DNS-01 チャレンジ設定 dnsChallenge: # Cloudflare プロバイダーを使用 provider: cloudflare # DNS レコード伝播の待機時間（秒） # Cloudflareは通常速いため短めでOK delayBeforeCheck: \u0026#34;10s\u0026#34; # 使用するDNSリゾルバー（任意） resolvers: - \u0026#34;1.1.1.1:53\u0026#34; - \u0026#34;8.8.8.8:53\u0026#34; # ワイルドカード証明書を取得する場合はこれが必須 # 環境変数 CF_API_EMAIL, CF_DNS_API_TOKEN が必要 # ------------------------------------------------------------ # 証明書の自動更新設定 # ------------------------------------------------------------ # 証明書の有効期限チェック間隔（デフォルト: 24時間） # 1日1回、証明書の有効期限をチェック #certificatesDuration: 2160 # 証明書の有効期限（時間）デフォルト90日 # ------------------------------------------------------------ # プロバイダー設定 # ------------------------------------------------------------ # 動的設定の読み込み元を指定します providers: # ファイルプロバイダー（動的設定ファイル） #file: # 動的設定ファイルのパス #filename: \u0026#34;/etc/traefik/dynamic.yml\u0026#34; # または設定ファイルが格納されたディレクトリを指定 # directory: \u0026#34;/etc/traefik/config/\u0026#34; # ファイル変更の監視 #watch: true # Docker プロバイダー docker: # Docker ソケットへのパス endpoint: \u0026#34;unix:///var/run/docker.sock\u0026#34; # 明示的にラベル設定されたコンテナのみ公開 exposedByDefault: false # Docker ネットワーク名 network: traefik-network # Swarm モードを有効にする場合 # swarmMode: true # ------------------------------------------------------------ # ヘルスチェック / Ping # ------------------------------------------------------------ ping: # ヘルスチェックエンドポイントのエントリーポイント entryPoint: web # ------------------------------------------------------------ # サーバートランスポート設定 # ------------------------------------------------------------ serversTransport: # バックエンドへの接続時にTLS証明書を検証しない（開発環境用） insecureSkipVerify: false # ルート証明書の追加 # rootCAs: # - \u0026#34;/etc/traefik/certs/ca.crt\u0026#34; # 最大アイドル接続数 maxIdleConnsPerHost: 200 dynamic.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 # ------------------------------------------------------------ # HTTP 設定 # ------------------------------------------------------------ http: # ========================================================== # ルーター設定 # ========================================================== # ルーターはリクエストをサービスに振り分けるルールを定義します routers: # ---------------------------------------------------------- # 例1: シンプルなWebアプリケーション # ---------------------------------------------------------- webapp: # ルールの定義（Host, PathPrefix, Headers などを組み合わせ可能） rule: \u0026#34;Host(`*.example.dev`)\u0026#34; # 使用するエントリーポイント entryPoints: - websecure # 振り分け先のサービス service: webapp-service # 適用するミドルウェア（複数指定可能） middlewares: - security-headers # TLS設定 tls: certResolver: letsencrypt acme.json\n上記ファイルは空ファイルを作成して下記コマンドを実行して権限を付与する\n※SSLの証明書情報はファイルの中に記載される 1 2 touch acme.json chmod 600 acme.json compose-nginx.yaml\nサンプルアプリケーションを想定 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 services: nginx: image: nginx:1.29.4 container_name: nginx-sample restart: unless-stopped networks: - traefik-network # カスタムHTMLを配置する場合は以下のコメントを解除 # volumes: # - ./html:/usr/share/nginx/html:ro labels: # Traefik で公開する - \u0026#34;traefik.enable=true\u0026#34; # ルーター設定（実際のドメインに変更してください） - \u0026#34;traefik.http.routers.nginx.rule=Host(`nginx.example.dev`)\u0026#34; - \u0026#34;traefik.http.routers.nginx.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.nginx.tls.certresolver=letsencrypt\u0026#34; # ワイルドカード証明書の設定 - \u0026#34;traefik.http.routers.nginx.tls.domains[0].main=example.dev\u0026#34; - \u0026#34;traefik.http.routers.nginx.tls.domains[0].sans=*.example.dev\u0026#34; # サービス設定 - \u0026#34;traefik.http.services.nginx.loadbalancer.server.port=80\u0026#34; # ミドルウェア適用 - \u0026#34;traefik.http.routers.nginx.middlewares=security-headers@file\u0026#34; networks: # 外部ネットワークとしてTraefikのネットワークを参照 traefik-network: external: true 起動 下記コマンドを実行する\n1 2 sudo docker compose up -d sudo docker compose down -v Dashboardへは下記のようなURLからアクセスできる\n\u0026ldquo;http://IP:8080/dashboard/\u0026rdquo; SSL証明書 この時点では\u0026quot;acme.json\u0026quot;に証明書情報は記載されない\nSSL証明書を使用するアプリケーションを追加した際に記載される\nコマンドを実行してnginxを起動します\n1 docker compose -f compose.nginx.yaml up -d DNSにドメイン名とIPを登録してhttpsでアクセスできるか確認します\nアクセスできれば問題なく実行できています\nアクセスする際に信頼できない証明書のエラーがでるが、ステージング用の証明書を使用しているので、一旦無視する\n本番環境のSSL証明書を使用する \u0026ldquo;traefik.yaml\u0026quot;に記載されているステージング用をコメントアウトして、本番用のコメントアウトを解除する\nステージング用 1 2 3 4 # 本番用エンドポイント（テスト完了後にこちらを使用） # caServer: \u0026#34;https://acme-v02.api.letsencrypt.org/directory\u0026#34; # テスト用エンドポイント（レート制限回避、最初はこちらで動作確認） caServer: \u0026#34;https://acme-staging-v02.api.letsencrypt.org/directory\u0026#34; 本番用 1 2 3 4 # 本番用エンドポイント（テスト完了後にこちらを使用） caServer: \u0026#34;https://acme-v02.api.letsencrypt.org/directory\u0026#34; # テスト用エンドポイント（レート制限回避、最初はこちらで動作確認） # caServer: \u0026#34;https://acme-staging-v02.api.letsencrypt.org/directory\u0026#34; これで警告が出ることなく接続できます\nあとは必要なアプリケーションを実行していけばhttpsで接続できるようになります\n参考URL https://tech.natsuneko.blog/entry/2024/01/09/generate-lets-encrypt-dns-challenge-tls-certificate-via-cloudflare-dns-using-traefik https://doc.traefik.io/traefik/reference/install-configuration/tls/certificate-resolvers/acme/ https://note.com/dtreaction/n/n4202c0b6c1f0 https://go-acme.github.io/lego/dns/cloudflare/index.html https://qiita.com/everylittle/items/ae0a72658fea3061f129 https://zenn.dev/shimakaze_soft/articles/a91d9ec8b14a21 ","date":"2026-01-30T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/traefik-cloudflare-ssl/","title":"TraefikとCloudflareでワイルドカードSSL証明書を発行する"},{"content":"2024年振り返り 2025年の目標として掲げていた「応用情報技術者試験合格」、「LPIC1合格」、「1つ山に登る」、「自分で基板から作ったIoT関係のものを作る」はどれも達成できなかった。\n一つは意志が弱くそこまで本気でやらない状況が続いてしまった。\nあとは計画を何も立てずに過ごしてしまった、かつその状況に危機感を覚えずに過ごしてしまったこと。\nできたこととしては、IoT関係の部分は回路の仮組みまではできている状況なので引き続き継続して進めていきたい。\n応用情報はチャレンジしたが不合格となってしまった、前回の反省からしっかりと勉強したが合格には届かなかった。\n2025年に目標までではないが、したいこととして上げていた「Kubernetes上でアプリケーションを動かす」に関しては全くKubernetesに触ってすらいなかった。\nブログ記事の数に関しても年の後半に全然書かなくなってしまい15個しか作成できなかった。\n2026年こそは30個を超える記事をアプトプットしていきたい。\n2026年の目標 2026年の目標は「応用情報技術者試験合格」、「FP3級合格」、「高尾山に登る」、「温湿度CO2濃度センサー基板を作ってWEB画面から見れるようにする」、「AtCoderで緑以上になる」を頑張っていきます。\n応用情報技術者試験合格 IT全般の知識を身に着けたい FP3級合格 資産に関する知識のため 高尾山に登る 体力作りのため 温湿度CO2濃度センサー基板を作ってWEB画面から見れるようにする 部屋の状況管理と電子工作が好きだから AtCoderで緑以上になる アルゴリズムやプログラムの力を身につけるため ","date":"2026-01-04T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/2025review-and-2026plan/","title":"2025年振り返りと2026年の目標"},{"content":"参考URL 基本は公式の手順に従ってアップグレードする\nUpgrade from 8 to 9 https://pve.proxmox.com/wiki/Upgrade_from_8_to_9 Proxmox VE 8 を9にアップグレードしたときのメモ https://zenn.dev/omohikane/articles/upgrade-proxmoxve8to9 PVE 9.0アップデート手順のリポジトリ設定がわからなかった話（aptの.listと.source） https://qiita.com/oishi-d/items/2618fbb3a87520737b94 現在の状態 \u0026ldquo;neofetch\u0026quot;で確認する\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 root@pve:~# neofetch .://:` `://:. root@pve `hMMMMMMd/ /dMMMMMMh` -------- `sMMMMMMMd: :mMMMMMMMs` OS: Proxmox VE 8.4.11 x86_64 `-/+oo+/:`.yMMMMMMMh- -hMMMMMMMy.`:/+oo+/-` Kernel: 6.8.12-8-pve `:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:` Uptime: 159 days, 4 hours, 14 mins `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/` Packages: 873 (dpkg) ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/. Shell: bash 5.2.15 .+ooooooo+-`oNMMMMNo`-+ooooooo+. Terminal: /dev/pts/0 -+ooooooo/.`sMMs`./ooooooo+- CPU: AMD Ryzen 7 5700G with Radeon Graphics (16) @ 4.673GHz :oooooooo/`..`/oooooooo: GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile Series :oooooooo/`..`/oooooooo: Memory: 2082MiB / 60133MiB -+ooooooo/.`sMMs`./ooooooo+- .+ooooooo+-`oNMMMMNo`-+ooooooo+. ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/. `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/` `:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:` `-/+oo+/:`.yMMMMMMMh- -hMMMMMMMy.`:/+oo+/-` `sMMMMMMMm: :dMMMMMMMs` `hMMMMMMd/ /dMMMMMMh` `://:` `://:` root@pve:~# アップグレードの事前確認をする SSHでProxmoxのマシンに接続します\n下記コマンドを実行してチェックをします\n1 pve8to9 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 root@pve:~# pve8to9 perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LC_ADDRESS = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_NAME = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_MONETARY = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_PAPER = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_IDENTIFICATION = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_TELEPHONE = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_MEASUREMENT = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_TIME = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_NUMERIC = \u0026#34;ja_JP.UTF-8\u0026#34;, LANG = \u0026#34;en_US.UTF-8\u0026#34; are supported and installed on your system. perl: warning: Falling back to a fallback locale (\u0026#34;en_US.UTF-8\u0026#34;). = CHECKING VERSION INFORMATION FOR PVE PACKAGES = Checking for package updates.. PASS: all packages up-to-date Checking proxmox-ve package version.. PASS: proxmox-ve package has version \u0026gt;= 8.4-0 Checking running kernel version.. PASS: running kernel \u0026#39;6.8.12-8-pve\u0026#39; is considered suitable for upgrade. = CHECKING CLUSTER HEALTH/SETTINGS = SKIP: standalone node. = CHECKING HYPER-CONVERGED CEPH STATUS = SKIP: no hyper-converged ceph setup detected! = CHECKING CONFIGURED STORAGES = PASS: storage \u0026#39;add-data\u0026#39; enabled and active. PASS: storage \u0026#39;local\u0026#39; enabled and active. PASS: storage \u0026#39;local-lvm\u0026#39; enabled and active. PASS: storage \u0026#39;vm-backup\u0026#39; enabled and active. INFO: Checking storage content type configuration.. PASS: no storage content problems found PASS: no storage re-uses a directory for multiple content types. INFO: Check for usage of native GlusterFS storage plugin... PASS: No GlusterFS storage found. INFO: Checking whether all external RBD storages have the \u0026#39;keyring\u0026#39; option configured SKIP: No RBD storage configured. = VIRTUAL GUEST CHECKS = INFO: Checking for running guests.. PASS: no running guest detected. INFO: Checking if LXCFS is running with FUSE3 library, if already upgraded.. SKIP: not yet upgraded, no need to check the FUSE library version LXCFS uses INFO: Checking for VirtIO devices that would change their MTU... PASS: All guest config descriptions fit in the new limit of 8 KiB INFO: Checking container configs for deprecated lxc.cgroup entries PASS: No legacy \u0026#39;lxc.cgroup\u0026#39; keys found. INFO: Checking VM configurations for outdated machine versions PASS: All VM machine versions are recent enough = MISCELLANEOUS CHECKS = INFO: Checking common daemon services.. PASS: systemd unit \u0026#39;pveproxy.service\u0026#39; is in state \u0026#39;active\u0026#39; PASS: systemd unit \u0026#39;pvedaemon.service\u0026#39; is in state \u0026#39;active\u0026#39; PASS: systemd unit \u0026#39;pvescheduler.service\u0026#39; is in state \u0026#39;active\u0026#39; PASS: systemd unit \u0026#39;pvestatd.service\u0026#39; is in state \u0026#39;active\u0026#39; INFO: Checking for supported \u0026amp; active NTP service.. PASS: Detected active time synchronisation unit \u0026#39;chrony.service\u0026#39; INFO: Checking if the local node\u0026#39;s hostname \u0026#39;pve\u0026#39; is resolvable.. INFO: Checking if resolved IP is configured on local node.. PASS: Resolved node IP \u0026#39;192.168.10.111\u0026#39; configured and active on single interface. INFO: Check node certificate\u0026#39;s RSA key size PASS: Certificate \u0026#39;pve-root-ca.pem\u0026#39; passed Debian Busters (and newer) security level for TLS connections (4096 \u0026gt;= 2048) PASS: Certificate \u0026#39;pve-ssl.pem\u0026#39; passed Debian Busters (and newer) security level for TLS connections (2048 \u0026gt;= 2048) INFO: Checking backup retention settings.. PASS: no backup retention problems found. INFO: checking CIFS credential location.. PASS: no CIFS credentials at outdated location found. INFO: Checking permission system changes.. INFO: Checking custom role IDs PASS: no custom roles defined INFO: Checking node and guest description/note length.. PASS: All node config descriptions fit in the new limit of 64 KiB INFO: Checking if the suite for the Debian security repository is correct.. PASS: found no suite mismatch INFO: Checking for existence of NVIDIA vGPU Manager.. PASS: No NVIDIA vGPU Service found. INFO: Checking bootloader configuration... PASS: bootloader packages installed correctly INFO: Check for dkms modules... SKIP: could not get dkms status INFO: Check for legacy \u0026#39;filter\u0026#39; or \u0026#39;group\u0026#39; sections in /etc/pve/notifications.cfg... INFO: Check for legacy \u0026#39;notification-policy\u0026#39; or \u0026#39;notification-target\u0026#39; options in /etc/pve/jobs.cfg... PASS: No legacy \u0026#39;notification-policy\u0026#39; or \u0026#39;notification-target\u0026#39; options found! INFO: Check for LVM autoactivation settings on LVM and LVM-thin storages... NOTICE: storage \u0026#39;add-data\u0026#39; has guest volumes with autoactivation enabled NOTICE: storage \u0026#39;local-lvm\u0026#39; has guest volumes with autoactivation enabled NOTICE: Starting with PVE 9, autoactivation will be disabled for new LVM/LVM-thin guest volumes. This system has some volumes that still have autoactivation enabled. All volumes with autoactivations reside on local storage, where this normally does not cause any issues. You can run the following command to disable autoactivation for existing LVM/LVM-thin guest volumes: /usr/share/pve-manager/migrations/pve-lvm-disable-autoactivation INFO: Checking lvm config for thin_check_options... PASS: Check for correct thin_check_options passed INFO: Check space requirements for RRD migration... PASS: Enough free disk space for increased RRD metric granularity requirements, which is roughly 74.66 MiB. INFO: Checking for IPAM DB files that have not yet been migrated. PASS: No legacy IPAM DB found. PASS: No legacy MAC DB found. INFO: Checking if the legacy sysctl file \u0026#39;/etc/sysctl.conf\u0026#39; needs to be migrated to new \u0026#39;/etc/sysctl.d/\u0026#39; path. PASS: Legacy file \u0026#39;/etc/sysctl.conf\u0026#39; exists but does not contain any settings. INFO: Checking if matching CPU microcode package is installed. WARN: The matching CPU microcode package \u0026#39;amd64-microcode\u0026#39; could not be found! Consider installing it to receive the latest security and bug fixes for your CPU. Ensure you enable the \u0026#39;non-free-firmware\u0026#39; component in the apt sources and run: apt install amd64-microcode SKIP: NOTE: Expensive checks, like CT cgroupv2 compat, not performed without \u0026#39;--full\u0026#39; parameter = SUMMARY = TOTAL: 45 PASSED: 35 SKIPPED: 6 WARNINGS: 1 FAILURES: 0 ATTENTION: Please check the output for detailed information! root@pve:~# フルのオプションを付けて確認する\n1 pve8to9 --full 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 root@pve:~# pve8to9 --full perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LC_ADDRESS = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_NAME = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_MONETARY = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_PAPER = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_IDENTIFICATION = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_TELEPHONE = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_MEASUREMENT = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_TIME = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_NUMERIC = \u0026#34;ja_JP.UTF-8\u0026#34;, LANG = \u0026#34;en_US.UTF-8\u0026#34; are supported and installed on your system. perl: warning: Falling back to a fallback locale (\u0026#34;en_US.UTF-8\u0026#34;). = CHECKING VERSION INFORMATION FOR PVE PACKAGES = Checking for package updates.. PASS: all packages up-to-date Checking proxmox-ve package version.. PASS: proxmox-ve package has version \u0026gt;= 8.4-0 Checking running kernel version.. PASS: running kernel \u0026#39;6.8.12-8-pve\u0026#39; is considered suitable for upgrade. = CHECKING CLUSTER HEALTH/SETTINGS = SKIP: standalone node. = CHECKING HYPER-CONVERGED CEPH STATUS = SKIP: no hyper-converged ceph setup detected! = CHECKING CONFIGURED STORAGES = PASS: storage \u0026#39;add-data\u0026#39; enabled and active. PASS: storage \u0026#39;local\u0026#39; enabled and active. PASS: storage \u0026#39;local-lvm\u0026#39; enabled and active. PASS: storage \u0026#39;vm-backup\u0026#39; enabled and active. INFO: Checking storage content type configuration.. PASS: no storage content problems found PASS: no storage re-uses a directory for multiple content types. INFO: Check for usage of native GlusterFS storage plugin... PASS: No GlusterFS storage found. INFO: Checking whether all external RBD storages have the \u0026#39;keyring\u0026#39; option configured SKIP: No RBD storage configured. = VIRTUAL GUEST CHECKS = INFO: Checking for running guests.. PASS: no running guest detected. INFO: Checking if LXCFS is running with FUSE3 library, if already upgraded.. SKIP: not yet upgraded, no need to check the FUSE library version LXCFS uses INFO: Checking for VirtIO devices that would change their MTU... PASS: All guest config descriptions fit in the new limit of 8 KiB INFO: Checking container configs for deprecated lxc.cgroup entries PASS: No legacy \u0026#39;lxc.cgroup\u0026#39; keys found. INFO: Checking VM configurations for outdated machine versions PASS: All VM machine versions are recent enough = MISCELLANEOUS CHECKS = INFO: Checking common daemon services.. PASS: systemd unit \u0026#39;pveproxy.service\u0026#39; is in state \u0026#39;active\u0026#39; PASS: systemd unit \u0026#39;pvedaemon.service\u0026#39; is in state \u0026#39;active\u0026#39; PASS: systemd unit \u0026#39;pvescheduler.service\u0026#39; is in state \u0026#39;active\u0026#39; PASS: systemd unit \u0026#39;pvestatd.service\u0026#39; is in state \u0026#39;active\u0026#39; INFO: Checking for supported \u0026amp; active NTP service.. PASS: Detected active time synchronisation unit \u0026#39;chrony.service\u0026#39; INFO: Checking if the local node\u0026#39;s hostname \u0026#39;pve\u0026#39; is resolvable.. INFO: Checking if resolved IP is configured on local node.. PASS: Resolved node IP \u0026#39;192.168.10.111\u0026#39; configured and active on single interface. INFO: Check node certificate\u0026#39;s RSA key size PASS: Certificate \u0026#39;pve-root-ca.pem\u0026#39; passed Debian Busters (and newer) security level for TLS connections (4096 \u0026gt;= 2048) PASS: Certificate \u0026#39;pve-ssl.pem\u0026#39; passed Debian Busters (and newer) security level for TLS connections (2048 \u0026gt;= 2048) INFO: Checking backup retention settings.. PASS: no backup retention problems found. INFO: checking CIFS credential location.. PASS: no CIFS credentials at outdated location found. INFO: Checking permission system changes.. INFO: Checking custom role IDs PASS: no custom roles defined INFO: Checking node and guest description/note length.. PASS: All node config descriptions fit in the new limit of 64 KiB INFO: Checking if the suite for the Debian security repository is correct.. PASS: found no suite mismatch INFO: Checking for existence of NVIDIA vGPU Manager.. PASS: No NVIDIA vGPU Service found. INFO: Checking bootloader configuration... PASS: bootloader packages installed correctly INFO: Check for dkms modules... SKIP: could not get dkms status INFO: Check for legacy \u0026#39;filter\u0026#39; or \u0026#39;group\u0026#39; sections in /etc/pve/notifications.cfg... INFO: Check for legacy \u0026#39;notification-policy\u0026#39; or \u0026#39;notification-target\u0026#39; options in /etc/pve/jobs.cfg... PASS: No legacy \u0026#39;notification-policy\u0026#39; or \u0026#39;notification-target\u0026#39; options found! INFO: Check for LVM autoactivation settings on LVM and LVM-thin storages... NOTICE: storage \u0026#39;add-data\u0026#39; has guest volumes with autoactivation enabled NOTICE: storage \u0026#39;local-lvm\u0026#39; has guest volumes with autoactivation enabled NOTICE: Starting with PVE 9, autoactivation will be disabled for new LVM/LVM-thin guest volumes. This system has some volumes that still have autoactivation enabled. All volumes with autoactivations reside on local storage, where this normally does not cause any issues. You can run the following command to disable autoactivation for existing LVM/LVM-thin guest volumes: /usr/share/pve-manager/migrations/pve-lvm-disable-autoactivation INFO: Checking lvm config for thin_check_options... PASS: Check for correct thin_check_options passed INFO: Check space requirements for RRD migration... PASS: Enough free disk space for increased RRD metric granularity requirements, which is roughly 74.66 MiB. INFO: Checking for IPAM DB files that have not yet been migrated. PASS: No legacy IPAM DB found. PASS: No legacy MAC DB found. INFO: Checking if the legacy sysctl file \u0026#39;/etc/sysctl.conf\u0026#39; needs to be migrated to new \u0026#39;/etc/sysctl.d/\u0026#39; path. PASS: Legacy file \u0026#39;/etc/sysctl.conf\u0026#39; exists but does not contain any settings. INFO: Checking if matching CPU microcode package is installed. WARN: The matching CPU microcode package \u0026#39;amd64-microcode\u0026#39; could not be found! Consider installing it to receive the latest security and bug fixes for your CPU. Ensure you enable the \u0026#39;non-free-firmware\u0026#39; component in the apt sources and run: apt install amd64-microcode SKIP: No containers on node detected. = SUMMARY = TOTAL: 45 PASSED: 35 SKIPPED: 6 WARNINGS: 1 FAILURES: 0 ATTENTION: Please check the output for detailed information! root@pve:~# リポジトリを更新します 下記コマンドを実行します\n1 2 3 apt update apt dist-upgrade pveversion 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 root@pve:~# pveversion perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LC_ADDRESS = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_NAME = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_MONETARY = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_PAPER = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_IDENTIFICATION = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_TELEPHONE = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_MEASUREMENT = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_TIME = \u0026#34;ja_JP.UTF-8\u0026#34;, LC_NUMERIC = \u0026#34;ja_JP.UTF-8\u0026#34;, LANG = \u0026#34;en_US.UTF-8\u0026#34; are supported and installed on your system. perl: warning: Falling back to a fallback locale (\u0026#34;en_US.UTF-8\u0026#34;). pve-manager/8.4.11/14a32011146091ed (running kernel: 6.8.12-8-pve) root@pve:~# リポジトリを変更します 下記コマンド実行します\n1 2 sed -i \u0026#39;s/bookworm/trixie/g\u0026#39; /etc/apt/sources.list sed -i \u0026#39;s/bookworm/trixie/g\u0026#39; /etc/apt/sources.list.d/pve-enterprise.list 1 2 cat /etc/apt/sources.list cat /etc/apt/sources.list.d/pve-enterprise.list 1 2 3 4 5 6 7 8 9 10 11 12 13 14 root@pve:~# cat /etc/apt/sources.list deb http://ftp.jp.debian.org/debian trixie main contrib deb http://ftp.jp.debian.org/debian trixie-updates main contrib # security updates deb http://security.debian.org trixie-security main contrib deb http://download.proxmox.com/debian/pve trixie pve-no-subscription root@pve:~# cat /etc/apt/sources.list.d/pve-enterprise.list # deb https://enterprise.proxmox.com/debian/pve trixie pve-enterprise root@pve:~# パッケージリポジトリを追加する\n1 2 3 4 5 6 7 cat \u0026gt; /etc/apt/sources.list.d/proxmox.sources \u0026lt;\u0026lt; EOF Types: deb URIs: http://download.proxmox.com/debian/pve Suites: trixie Components: pve-no-subscription Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg EOF アップデートをすると重複エラーが表示される\n1 apt update 1 2 3 4 5 6 W: Target Packages (pve-no-subscription/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:8 and /etc/apt/sources.list.d/proxmox.sources:1 W: Target Packages (pve-no-subscription/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:8 and /etc/apt/sources.list.d/proxmox.sources:1 W: Target Translations (pve-no-subscription/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:8 and /etc/apt/sources.list.d/proxmox.sources:1 W: Target Packages (pve-no-subscription/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:8 and /etc/apt/sources.list.d/proxmox.sources:1 W: Target Packages (pve-no-subscription/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:8 and /etc/apt/sources.list.d/proxmox.sources:1 W: Target Translations (pve-no-subscription/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:8 and /etc/apt/sources.list.d/proxmox.sources:1 エラーを解消する \u0026ldquo;apt modernize-sources\u0026quot;コマンド実行する\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 root@pve:~# apt modernize-sources The following files need modernizing: - /etc/apt/sources.list - /etc/apt/sources.list.d/ceph.list - /etc/apt/sources.list.d/pve-enterprise.list Modernizing will replace .list files with the new .sources format, add Signed-By values where they can be determined automatically, and save the old files into .list.bak files. This command supports the \u0026#39;signed-by\u0026#39; and \u0026#39;trusted\u0026#39; options. If you have specified other options inside [] brackets, please transfer them manually to the output files; see sources.list(5) for a mapping. For a simulation, respond N in the following prompt. Rewrite 3 sources? [Y/n] Y Modernizing /etc/apt/sources.list... - Writing /etc/apt/sources.list.d/debian.sources - Writing /etc/apt/sources.list.d/proxmox.sources Modernizing /etc/apt/sources.list.d/ceph.list... Modernizing /etc/apt/sources.list.d/pve-enterprise.list... root@pve:~# エラーがでなくなった\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 root@pve:~# apt update Hit:1 http://security.debian.org trixie-security InRelease Hit:2 http://ftp.jp.debian.org/debian trixie InRelease Hit:3 http://ftp.jp.debian.org/debian trixie-updates InRelease Hit:4 http://download.proxmox.com/debian/pve trixie InRelease 1 package can be upgraded. Run \u0026#39;apt list --upgradable\u0026#39; to see it. root@pve:~# nano /etc/apt/sources.list root@pve:~# nano /etc/apt/sources.list.d/proxmox.sources root@pve:~# apt update Hit:1 http://ftp.jp.debian.org/debian trixie InRelease Hit:2 http://ftp.jp.debian.org/debian trixie-updates InRelease Hit:3 http://security.debian.org trixie-security InRelease Hit:4 http://download.proxmox.com/debian/pve trixie InRelease 1 package can be upgraded. Run \u0026#39;apt list --upgradable\u0026#39; to see it. root@pve:~# Proxmox9へアップグレードする 下記コマンドを実行します\n1 2 apt update apt dist-upgrade 何回か選択を迫られるので下記の選択をした\n1 2 3 q N N アップグレードが終わったら再起動をする\nトラブル 起動しなくなった、BIOSの画面になってしまう\nBIOS画面から、直接起動ディスクを指定してもBIOSの画面になってしまう\nProxmoxのインストールUSBを作成して、USBメモリを差しインストーラーを起動する際に\u0026quot;Boot Rescue\u0026quot;を選択し起動できた\nVMをバックアップする\nGRUBを再インストールする \u0026ldquo;lsblk\u0026quot;でgrubが入っているディスクを探す\n\u0026ldquo;/boot/efi\u0026quot;が入っているパーティションを探す 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 root@pve:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 465.8G 0 disk ├─sda1 8:1 0 1007K 0 part ├─sda2 8:2 0 1G 0 part /boot/efi └─sda3 8:3 0 464.8G 0 part ├─pve-swap 252:2 0 8G 0 lvm [SWAP] ├─pve-root 252:3 0 96G 0 lvm / ├─pve-data_tmeta 252:4 0 3.4G 0 lvm │ └─pve-data-tpool 252:6 0 337.9G 0 lvm │ ├─pve-data 252:7 0 337.9G 1 lvm │ ├─pve-vm--101--disk--0 252:8 0 32G 0 lvm │ ├─pve-vm--102--disk--0 252:10 0 32G 0 lvm │ ├─pve-vm--102--disk--1 252:13 0 32G 0 lvm │ ├─pve-vm--105--disk--0 252:15 0 32G 0 lvm │ ├─pve-vm--105--disk--1 252:17 0 50G 0 lvm │ ├─pve-vm--111--disk--0 252:19 0 32G 0 lvm │ ├─pve-vm--112--disk--0 252:21 0 32G 0 lvm │ ├─pve-vm--113--disk--0 252:23 0 32G 0 lvm │ ├─pve-vm--114--disk--0 252:25 0 32G 0 lvm │ ├─pve-vm--115--disk--0 252:27 0 32G 0 lvm │ ├─pve-vm--116--disk--0 252:29 0 10G 0 lvm │ ├─pve-vm--117--disk--0 252:32 0 32G 0 lvm │ ├─pve-vm--120--disk--0 252:34 0 10G 0 lvm │ ├─pve-vm--121--disk--0 252:36 0 10G 0 lvm │ ├─pve-vm--122--disk--0 252:38 0 10G 0 lvm │ ├─pve-vm--129--disk--0 252:40 0 10G 0 lvm │ ├─pve-vm--128--disk--0 252:42 0 20G 0 lvm │ ├─pve-vm--127--disk--0 252:44 0 20G 0 lvm │ ├─pve-vm--126--disk--0 252:45 0 20G 0 lvm │ ├─pve-vm--131--disk--0 252:46 0 20G 0 lvm │ ├─pve-vm--132--disk--0 252:47 0 20G 0 lvm │ ├─pve-vm--133--disk--0 252:48 0 20G 0 lvm │ ├─pve-vm--134--disk--0 252:49 0 20G 0 lvm │ └─pve-vm--135--disk--0 252:50 0 32G 0 lvm └─pve-data_tdata 252:5 0 337.9G 0 lvm └─pve-data-tpool 252:6 0 337.9G 0 lvm ├─pve-data 252:7 0 337.9G 1 lvm ├─pve-vm--101--disk--0 252:8 0 32G 0 lvm ├─pve-vm--102--disk--0 252:10 0 32G 0 lvm ├─pve-vm--102--disk--1 252:13 0 32G 0 lvm ├─pve-vm--105--disk--0 252:15 0 32G 0 lvm ├─pve-vm--105--disk--1 252:17 0 50G 0 lvm ├─pve-vm--111--disk--0 252:19 0 32G 0 lvm ├─pve-vm--112--disk--0 252:21 0 32G 0 lvm ├─pve-vm--113--disk--0 252:23 0 32G 0 lvm ├─pve-vm--114--disk--0 252:25 0 32G 0 lvm ├─pve-vm--115--disk--0 252:27 0 32G 0 lvm ├─pve-vm--116--disk--0 252:29 0 10G 0 lvm ├─pve-vm--117--disk--0 252:32 0 32G 0 lvm ├─pve-vm--120--disk--0 252:34 0 10G 0 lvm ├─pve-vm--121--disk--0 252:36 0 10G 0 lvm ├─pve-vm--122--disk--0 252:38 0 10G 0 lvm ├─pve-vm--129--disk--0 252:40 0 10G 0 lvm ├─pve-vm--128--disk--0 252:42 0 20G 0 lvm ├─pve-vm--127--disk--0 252:44 0 20G 0 lvm ├─pve-vm--126--disk--0 252:45 0 20G 0 lvm ├─pve-vm--131--disk--0 252:46 0 20G 0 lvm ├─pve-vm--132--disk--0 252:47 0 20G 0 lvm ├─pve-vm--133--disk--0 252:48 0 20G 0 lvm ├─pve-vm--134--disk--0 252:49 0 20G 0 lvm └─pve-vm--135--disk--0 252:50 0 32G 0 lvm sdb 8:16 0 3.6T 0 disk └─sdb1 8:17 0 3.6T 0 part /mnt/pve/vm-backup sdc 8:32 1 28.9G 0 disk ├─sdc1 8:33 1 282K 0 part ├─sdc2 8:34 1 8M 0 part ├─sdc3 8:35 1 1.5G 0 part └─sdc4 8:36 1 300K 0 part nvme0n1 259:0 0 931.5G 0 disk ├─add--data-add--data_tmeta 252:0 0 9.3G 0 lvm │ └─add--data-add--data-tpool 252:9 0 912.8G 0 lvm │ ├─add--data-add--data 252:11 0 912.8G 1 lvm │ ├─add--data-vm--109--disk--0 252:12 0 200G 0 lvm │ ├─add--data-vm--124--disk--0 252:14 0 32G 0 lvm │ ├─add--data-vm--130--disk--0 252:16 0 32G 0 lvm │ ├─add--data-vm--134--disk--0 252:18 0 20G 0 lvm │ ├─add--data-vm--133--disk--0 252:20 0 20G 0 lvm │ ├─add--data-vm--132--disk--0 252:22 0 20G 0 lvm │ ├─add--data-vm--136--disk--0 252:24 0 50G 0 lvm │ ├─add--data-vm--100--disk--0 252:26 0 70G 0 lvm │ ├─add--data-vm--123--disk--0 252:28 0 10G 0 lvm │ ├─add--data-vm--138--disk--0 252:30 0 32G 0 lvm │ ├─add--data-vm--144--disk--0 252:31 0 50G 0 lvm │ ├─add--data-vm--145--disk--0 252:33 0 32G 0 lvm │ ├─add--data-vm--146--disk--0 252:35 0 10G 0 lvm │ ├─add--data-vm--151--disk--0 252:37 0 32G 0 lvm │ ├─add--data-vm--200--disk--0 252:39 0 4M 0 lvm │ ├─add--data-vm--200--disk--1 252:41 0 100G 0 lvm │ └─add--data-vm--200--disk--2 252:43 0 4M 0 lvm └─add--data-add--data_tdata 252:1 0 912.8G 0 lvm └─add--data-add--data-tpool 252:9 0 912.8G 0 lvm ├─add--data-add--data 252:11 0 912.8G 1 lvm ├─add--data-vm--109--disk--0 252:12 0 200G 0 lvm ├─add--data-vm--124--disk--0 252:14 0 32G 0 lvm ├─add--data-vm--130--disk--0 252:16 0 32G 0 lvm ├─add--data-vm--134--disk--0 252:18 0 20G 0 lvm ├─add--data-vm--133--disk--0 252:20 0 20G 0 lvm ├─add--data-vm--132--disk--0 252:22 0 20G 0 lvm ├─add--data-vm--136--disk--0 252:24 0 50G 0 lvm ├─add--data-vm--100--disk--0 252:26 0 70G 0 lvm ├─add--data-vm--123--disk--0 252:28 0 10G 0 lvm ├─add--data-vm--138--disk--0 252:30 0 32G 0 lvm ├─add--data-vm--144--disk--0 252:31 0 50G 0 lvm ├─add--data-vm--145--disk--0 252:33 0 32G 0 lvm ├─add--data-vm--146--disk--0 252:35 0 10G 0 lvm ├─add--data-vm--151--disk--0 252:37 0 32G 0 lvm ├─add--data-vm--200--disk--0 252:39 0 4M 0 lvm ├─add--data-vm--200--disk--1 252:41 0 100G 0 lvm └─add--data-vm--200--disk--2 252:43 0 4M 0 lvm root@pve:~# bootのフォルダを確認する\n1 2 3 4 5 6 7 8 9 root@pve:~# ls -al /dev/sda2 brw-rw---- 1 root disk 8, 2 Aug 14 15:41 /dev/sda2 root@pve:~# ls -al /boot/efi/EFI/ total 16 drwxr-xr-x 4 root root 4096 May 27 2023 . drwxr-xr-x 3 root root 4096 Jan 1 1970 .. drwxr-xr-x 2 root root 4096 May 27 2023 BOOT drwxr-xr-x 2 root root 4096 May 27 2023 proxmox root@pve:~# GRUBを再インストールする\n1 2 update-grub grub-install /dev/sda2 上記実行後、問題なく起動できることを確認する\nアップグレード完了 \u0026ldquo;neofetch\u0026quot;で確認する\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 root@pve:~# neofetch .://:` `://:. root@pve `hMMMMMMd/ /dMMMMMMh` -------- `sMMMMMMMd: :mMMMMMMMs` OS: Proxmox VE 9.0.5 x86_64 `-/+oo+/:`.yMMMMMMMh- -hMMMMMMMy.`:/+oo+/-` Kernel: 6.14.8-2-pve `:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:` Uptime: 9 mins `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/` Packages: 987 (dpkg) ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/. Shell: bash 5.2.37 .+ooooooo+-`oNMMMMNo`-+ooooooo+. Resolution: 1920x1080 -+ooooooo/.`sMMs`./ooooooo+- Terminal: /dev/pts/0 :oooooooo/`..`/oooooooo: CPU: AMD Ryzen 7 5700G with Radeon Graphics (16) @ 4.673GHz :oooooooo/`..`/oooooooo: GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile Series -+ooooooo/.`sMMs`./ooooooo+- Memory: 1602MiB / 60137MiB .+ooooooo+-`oNMMMMNo`-+ooooooo+. ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/. `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/` `:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:` `-/+oo+/:`.yMMMMMMMh- -hMMMMMMMy.`:/+oo+/-` `sMMMMMMMm: :dMMMMMMMs` `hMMMMMMd/ /dMMMMMMh` `://:` `://:` root@pve:~# ","date":"2025-08-15T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/proxmox-upgrade-8to9/","title":"Proxmoxを8から9へバージョンアップする"},{"content":"\u0026ldquo;apt update\u0026quot;をしたらバージョンが上がっている旨のメッセージがでてきたので対処する Proxmoxで\u0026quot;apt update\u0026quot;をしたらバージョンが上がっている旨のメッセージがでてきたので対処する\n1 N: Repository \u0026#39;http://ftp.jp.debian.org/debian bookworm InRelease\u0026#39; changed its \u0026#39;Version\u0026#39; value from \u0026#39;12.7\u0026#39; to \u0026#39;12.8\u0026#39; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 root@pve:~# apt update Get:1 http://security.debian.org bookworm-security InRelease [48.0 kB] Get:2 http://security.debian.org bookworm-security/main amd64 Packages [190 kB] Get:3 http://ftp.jp.debian.org/debian bookworm InRelease [151 kB] Get:4 http://ftp.jp.debian.org/debian bookworm-updates InRelease [55.4 kB] Get:5 http://ftp.jp.debian.org/debian bookworm/main amd64 Packages [8,789 kB] Get:6 http://ftp.jp.debian.org/debian bookworm/main Translation-en [6,109 kB] Hit:7 http://download.proxmox.com/debian/pve bookworm InRelease Fetched 15.3 MB in 2s (9,564 kB/s) Reading package lists... Done Building dependency tree... Done Reading state information... Done 43 packages can be upgraded. Run \u0026#39;apt list --upgradable\u0026#39; to see them. N: Repository \u0026#39;http://ftp.jp.debian.org/debian bookworm InRelease\u0026#39; changed its \u0026#39;Version\u0026#39; value from \u0026#39;12.7\u0026#39; to \u0026#39;12.8\u0026#39; root@pve:~# 対応 下記コマンドで対応する\n1 apt --allow-releaseinfo-change update 1 2 3 4 5 6 7 8 9 10 root@pve:~# apt --allow-releaseinfo-change update Hit:1 http://security.debian.org bookworm-security InRelease Hit:2 http://ftp.jp.debian.org/debian bookworm InRelease Hit:3 http://ftp.jp.debian.org/debian bookworm-updates InRelease Hit:4 http://download.proxmox.com/debian/pve bookworm InRelease Reading package lists... Done Building dependency tree... Done Reading state information... Done 43 packages can be upgraded. Run \u0026#39;apt list --upgradable\u0026#39; to see them. root@pve:~# メッセージは表示されなくなった\n1 2 3 4 5 6 7 8 9 10 root@pve:~# apt update Hit:1 http://security.debian.org bookworm-security InRelease Hit:2 http://ftp.jp.debian.org/debian bookworm InRelease Hit:3 http://ftp.jp.debian.org/debian bookworm-updates InRelease Hit:4 http://download.proxmox.com/debian/pve bookworm InRelease Reading package lists... Done Building dependency tree... Done Reading state information... Done 43 packages can be upgraded. Run \u0026#39;apt list --upgradable\u0026#39; to see them. root@pve:~# ","date":"2025-07-07T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/apt-update-inrelease-version-up/","title":"apt updateをした際にバージョンが上がっている旨のメッセージがでてきた場合の対処"},{"content":"HPからAppImageをダウンロードする https://www.cursor.com/ja 下記のようなファイルがダウンロードされる\nCursor-0.48.7-x86_64.AppImage ダウンロードしたファイルを右クリックして\u0026quot;プロパティ\u0026quot;を選択する \u0026ldquo;Executable as Program\u0026quot;をオンにする\nダブルクリックしたが起動できなかったので、ターミナルから起動しようとしたらエラーが表示された\n1 2 3 4 5 6 7 8 9 mao@mao (x86_64) : /home/mao/Downloads \u0026gt; ./Cursor-0.48.7-x86_64.AppImage dlopen(): error loading libfuse.so.2 AppImages require FUSE to run. You might still be able to extract the contents of this AppImage if you run it with the --appimage-extract option. See https://github.com/AppImage/AppImageKit/wiki/FUSE for more information FUSEが必要で、インストールされていないから実行できないよう なので、インストールをする\n下記の方法でインストールするとシステムが破損するので注意 1 sudo apt install fuse 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 mao@mao (x86_64) : /home/mao/Downloads \u0026gt; sudo apt install fuse [sudo] password for mao: Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages were automatically installed and are no longer required: gir1.2-gnomeautoar-0.1 gir1.2-gnomedesktop-3.0 libcue2 libdecor-0-0 libdecor-0-plugin-1-gtk libdee-1.0-4 libei1 libexiv2-27 libfreerdp-server3-3 libfreerdp3-3 libgexiv2-2 libgsf-1-114 libgsf-1-common libntfs-3g89t64 libportal-gtk4-1 libportal1 libtotem-plparser-common libtotem-plparser18 libtracker-sparql-3.0-0 libtss2-tcti-libtpms0t64 libtss2-tcti-spi-helper0t64 libtss2-tctildr0t64 libunity-protocol-private0 libunity-scopes-json-def-desktop libunity9 libwinpr3-3 nautilus-data tracker tracker-extract tracker-miner-fs xwayland Use \u0026#39;sudo apt autoremove\u0026#39; to remove them. The following additional packages will be installed: libfuse2t64 The following packages will be REMOVED: fuse3 gnome-remote-desktop gnome-shell-extension-desktop-icons-ng gvfs-fuse nautilus ntfs-3g ubuntu-desktop-minimal ubuntu-session xdg-desktop-portal xdg-desktop-portal-gnome xdg-desktop-portal-gtk The following NEW packages will be installed: fuse libfuse2t64 0 upgraded, 2 newly installed, 11 to remove and 0 not upgraded. Need to get 117 kB of archives. After this operation, 6,632 kB disk space will be freed. Do you want to continue? [Y/n] y Get:1 http://archive.ubuntu.com/ubuntu noble/universe amd64 libfuse2t64 amd64 2.9.9-8.1build1 [89.9 kB] Get:2 http://archive.ubuntu.com/ubuntu noble/universe amd64 fuse amd64 2.9.9-8.1build1 [27.6 kB] Fetched 117 kB in 1s (110 kB/s) (Reading database ... 198533 files and directories currently installed.) Removing gnome-remote-desktop (46.3-0ubuntu1) ... Selecting previously unselected package libfuse2t64:amd64. (Reading database ... 198506 files and directories currently installed.) Preparing to unpack .../libfuse2t64_2.9.9-8.1build1_amd64.deb ... Unpacking libfuse2t64:amd64 (2.9.9-8.1build1) ... (Reading database ... 198519 files and directories currently installed.) Removing ubuntu-desktop-minimal (1.539.2) ... Removing ubuntu-session (46.0-1ubuntu4) ... Removing xdg-desktop-portal-gnome (46.2-0ubuntu1) ... dpkg: xdg-desktop-portal: dependency problems, but removing anyway as you reques ted: xdg-desktop-portal-gtk depends on xdg-desktop-portal (\u0026gt;= 1.14.0). Removing xdg-desktop-portal (1.18.4-1ubuntu2.24.04.1) ... dpkg: fuse3: dependency problems, but removing anyway as you requested: snapd depends on fuse3 (\u0026gt;= 3.10.5-1) | fuse; however: Package fuse3 is to be removed. Package fuse is not installed. Package fuse3 which provides fuse is to be removed. ntfs-3g depends on fuse3. gvfs-fuse depends on fuse3. snapd depends on fuse3 (\u0026gt;= 3.10.5-1) | fuse; however: Package fuse3 is to be removed. Package fuse is not installed. Package fuse3 which provides fuse is to be removed. Removing fuse3 (3.14.0-5build1) ... update-initramfs: deferring update (trigger activated) dpkg: nautilus: dependency problems, but removing anyway as you requested: gnome-shell-extension-desktop-icons-ng depends on nautilus (\u0026gt;= 3.38). Removing nautilus (1:46.2-0ubuntu0.3) ... Selecting previously unselected package fuse. (Reading database ... 198439 files and directories currently installed.) Preparing to unpack .../fuse_2.9.9-8.1build1_amd64.deb ... Unpacking fuse (2.9.9-8.1build1) ... (Reading database ... 198449 files and directories currently installed.) Removing gnome-shell-extension-desktop-icons-ng (46+really47.0.9-1ubuntu1) ... Removing gvfs-fuse (1.54.0-1ubuntu2) ... Removing ntfs-3g (1:2022.10.3-1.2ubuntu3) ... Removing xdg-desktop-portal-gtk (1.15.1-1build2) ... Setting up libfuse2t64:amd64 (2.9.9-8.1build1) ... Setting up fuse (2.9.9-8.1build1) ... Installing new version of config file /etc/fuse.conf ... update-initramfs: deferring update (trigger activated) Processing triggers for initramfs-tools (0.142ubuntu25.5) ... update-initramfs: Generating /boot/initrd.img-6.8.0-57-generic Processing triggers for gnome-menus (3.36.0-1.1ubuntu3) ... Processing triggers for libc-bin (2.39-0ubuntu8.4) ... Processing triggers for man-db (2.12.0-4build2) ... Processing triggers for libglib2.0-0t64:amd64 (2.80.0-6ubuntu3.2) ... Processing triggers for dbus (1.14.10-4ubuntu4.1) ... Processing triggers for mailcap (3.70+nmu1ubuntu1) ... Processing triggers for desktop-file-utils (0.27-2build1) ... mao@mao (x86_64) : /home/mao/Downloads \u0026gt; 下記の方法でインストールすると大丈夫 https://github.com/AppImage/AppImageKit/wiki/FUSE 1 sudo apt install libfuse2t64 ダブルクリックして起動する 設定ファイルは下記の場所に生成される\n1 2 /home/user/.cursor /home/user/.config/Cursor ","date":"2025-07-06T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/cursor-install/","title":"Cursorのインストール方法のメモ"},{"content":"環境 Ubuntu 24.04.2 LTS Linux 6.8.0-60-generic Intel® Core™ i5-13500 × 20 Intel® UHD Graphics 770 (ADL-S GT1) \u0026ldquo;ffmpeg\u0026quot;のインストールされているか確認する 1 2 sudo apt list ffmpeg --installed sudo apt list ffmpeg 1 2 3 4 5 6 \u0026gt; sudo apt list ffmpeg --installed [sudo] password for mao: Listing... Done \u0026gt; sudo apt list ffmpeg Listing... Done ffmpeg/noble 7:6.1.1-3ubuntu5 amd64 \u0026ldquo;ffmpeg\u0026quot;のインストールをする 1 sudo apt install ffmpeg \u0026ldquo;OBS\u0026quot;をインストールする 1 2 3 sudo add-apt-repository ppa:obsproject/obs-studio sudo apt update sudo apt install obs-studio 設定の変更 \u0026ldquo;Settings\u0026quot;をクリックして、\u0026ldquo;Output\u0026quot;の\u0026quot;Output Mode\u0026quot;を\u0026quot;Advanced\u0026quot;にする ファイルの保存場所を変更する\n\u0026ldquo;Recording\u0026quot;タブの\u0026quot;Recording Path\u0026quot;を自分の保存したい場所にする 保存時の拡張子を変更する\n\u0026ldquo;Recording Format\u0026quot;を\u0026quot;MPEG-4(.mp4)\u0026ldquo;にする エンコーダーを変更する\n\u0026ldquo;Video Encoder\u0026quot;を\u0026quot;x264\u0026quot;にする エラーになって録画ができない\nhttps://obsproject.com/forum/threads/trying-to-record-results-in-couldnt-initialize-muxer-os_process_pipe_write-for-info-structure-failed.182163/ \u0026ldquo;Settings\u0026quot;をクリックして、\u0026ldquo;Audio\u0026quot;の\u0026quot;Sample Rate\u0026quot;を\u0026quot;44.1 kHz\u0026quot;にする 参考URL https://obsproject.com/ja/download ","date":"2025-05-25T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/ubuntu-obs-install/","title":"Ubuntu24.04.2LTSにOBSをインストールする"},{"content":"環境 ファームウェア RPI_PICO2-20250415-v1.25.0.uf2 Thonny 4.1.7 Raspberry Pi Pico(ラズパイピコ) Windows11 Pro 24H2 Proxmox上からUSBポートをパススルー設定をする 本体のUSBに差し込みます\n対象のVMの「ハードウェア」に「追加」＞「USBデバイス」＞「USBポートを使用」から製造者が「Raspberry Pi」になっているポートを選択します 「追加」を押します すると一覧にUSBデバイスが追加されます\nWIndows11上から接続されているか確認する Windows11上でデバイスマネージャーを開き「ほかのデバイス」に「RP2 Boot」が表示されていれば認識されています エクスプローラー側からUSBメモリ等と同じように「RPI-RP2」として認識されているかを確認する Thonnyをインストールする 今回はzipをダウンロードして使用します\n下記のURLからダウンロードします\nhttps://thonny.org/ ダウンロードしたファイルを展開します\nファイルの中にある「thonny.exe」をダブルクリックして起動します\nこの段階ではまだ認識されていません\nプログラムを書き込めるようにする準備 ラズパイピコにファームウェアをアップロードします\nファームウェアは下記のURLからダウンロードします\nhttps://www.raspberrypi.com/documentation/microcontrollers/micropython.html#drag-and-drop-micropython 今回は「Pico」を選択します 拡張子は「.uf2」 ラズパピピコがUSBメモリと同じように認識されているので、ダウンロードしたファームウェアをコピーします\nすると自動的に再認識されます\nthonny上の右下から「Raspberry Pi Pico」を選択します\nこれでプログラムが書き込めるようになりました プログラムを書き込み ラズパイピコ上のLEDを0.5秒間隔で点滅させるプログラムです\n1 2 3 4 5 6 7 8 9 10 from machine import Pin from time import sleep led = Pin(\u0026#39;LED\u0026#39;, Pin.OUT) while True: led.on() sleep(0.5) led.off() sleep(0.5) 「ファイル」「保存」を押すと保存先が表示されるので「Raspberry Pi Pico」を選択します\n「実行」「現在のスクリプトを実行」を押すと書き込んだプログラムが動きLEDが点滅します\n","date":"2025-05-06T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/windows11-on-proxmox-usb-passthrough/","title":"Proxmox上のWindows11にUSBパススルーをしてラズパイピコにプログラムを書き込む"},{"content":"インストール リポジトリを追加してインストールする\n1 2 3 sudo add-apt-repository ppa:wireshark-dev/stable sudo apt update sudo apt install wireshark 参考URL\nhttps://launchpad.net/~wireshark-dev/+archive/ubuntu/stable ユーザーをグループに追加する 下記コマンドで\u0026quot;wireshark\u0026quot;のグループにユーザーを追加する\n1 sudo usermod -aG wireshark *** ***：ユーザー名 起動する キャプチャしたいネットワークを選択する ","date":"2025-03-27T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/wireshark-install/","title":"WireSharkをUbuntuDesktopで使う"},{"content":"対処方法 Proxmoxの設定ファイルから、残ってしまったストレージを削除する\n1 nano /etc/pve/storage.cfg 下記のようなになっているので必要のない部分を削除する\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 dir: local path /var/lib/vz content backup,iso,vztmpl lvmthin: local-lvm thinpool data vgname pve content images,rootdir lvmthin: add-data thinpool add-data vgname add-data content rootdir,images nodes pve lvm: hdd-storage vgname hdd-storage content rootdir,images nodes pve shared 0 dir: vm-backup path /mnt/pve/vm-backup content snippets,iso,backup,images,vztmpl,rootdir is_mountpoint 1 nodes pve 参考URL https://pve.proxmox.com/wiki/Storage https://zaki-lknr.github.io/Misc/proxmox/ ","date":"2025-03-25T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/proxmox-delete-storage-webui/","title":"ProxmoxのWebUI上で削除したストレージがはてなマークのまま残ってしまった場合の対処方法"},{"content":"環境 Promox 8.3.0 kernel: 6.8.12-8-pve pve-manager: 8.3.4 Windows11 24H2 virtio-win-0.1.266 Hyper-Vで仮想マシンの作成をする インストールするOSのISOイメージをダウンロードしておきます\n今回はUbuntu24.04.2を使用します\n\u0026ldquo;Hyper-Vマネージャ\u0026quot;を起動します\n\u0026ldquo;新規\u0026rdquo;\u0026gt;\u0026ldquo;仮想マシン\u0026quot;をクリックします 次へをクリックします \u0026ldquo;第2世代\u0026quot;を選択肢します メモリを割り当てます ネットワークは一旦接続せずに使用します \u0026ldquo;仮想ハードディスクを作成する\u0026quot;を選択します\n名前、場所、容量を設定します \u0026ldquo;ブートCD/DVD-ROMからオペレーティングシステムをインストールする\u0026quot;を選択し、\u0026ldquo;イメージファイル\u0026quot;の右の\u0026quot;参照\u0026quot;をクリックしてISOファイルを選択します 確認の画面になり、問題なければ\u0026quot;完了\u0026quot;をクリックします 名前が\u0026quot;新しい仮想マシン\u0026quot;となっているので右クリックして名前を変更します 右クリックまたは右のメニューから\u0026quot;起動\u0026quot;をクリックします 右のメニューの接続をクリックします\n起動できていません 一度\u0026quot;停止\u0026quot;をクリックして仮想マシンを止めます\n右クリックをして\u0026quot;設定\u0026quot;をクリックします\n\u0026ldquo;セキュリティ\u0026quot;の\u0026quot;セキュアブートを有効にする\u0026quot;のチェックを外し\u0026quot;適用\u0026quot;をクリックします 起動させます\n右のメニューの\u0026quot;接続\u0026quot;をクリックすると仮想マシンの画面が表示されます\nUbuntuのインストール画面が表示されているので、通常のUbuntuと同じようにインストールを進めます インストールが終われば完了です 参考URL Proxmox VEの構築とVMをたてるまで https://zenn.dev/no292nukegara/articles/81f2cb6af34772 Proxmox VE7.xでwindows11VMを作成する https://zenn.dev/northeggman/articles/49c6b73c03c81c Proxmox VE で Windows 11 VM を立てる https://akam1o.hatenablog.jp/entry/2024/03/05/234501 ProxmoxでVMコンソールの解像度を変更する https://qiita.com/minoden_works/items/a4c94eb7728c946d3efe ","date":"2025-03-25T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/proxmox-windows11-hyper-v-2/","title":"Proxmox上にWindows11をインストールしてHyper-V使えるようにする（2/2）"},{"content":"環境 Promox 8.3.0 kernel: 6.8.12-8-pve pve-manager: 8.3.4 Windows11 24H2 virtio-win-0.1.266 準備 Windows11\nhttps://www.microsoft.com/ja-jp/software-download/windows11 virtio（vitrio-win-0.1.266.iso）\nhttps://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/ 上記は両方ともPromoxにISOとしてアップロードする\nWindows11のVM構築 ISOイメージはWindows11を選択する\nゲストOSを\u0026quot;Microsoft Windows\u0026quot;にし、バージョンを\u0026quot;11/2022/2025\u0026quot;にする\nその下にある\u0026quot;VirtIOドライバ用の追加ドライブを追加\u0026quot;にチェックを入れる\nISOイメージを選択する（\u0026ldquo;vitrio-win-0.1.266.iso\u0026rdquo;）\n\u0026ldquo;EFIストレージ\u0026quot;を保存する場所を選択する\n\u0026ldquo;Qemuエージェント\u0026quot;にチェックを入れる\n\u0026ldquo;TPM追加\u0026quot;にチェックを入れる\n\u0026ldquo;TPMストレージ\u0026quot;を保存する場所を選択する\n\u0026ldquo;バージョン\u0026quot;は\u0026quot;v2.0\u0026quot;を選択する\nディスクは100GBくらい確保しておく CPUの項目では\u0026quot;種別\u0026quot;を\u0026quot;host\u0026quot;にする メモリは16GBにする ネットワークはそのまま 確認に画面になり、問題なければ\u0026quot;完了\u0026quot;を押す Windows11インストール 設定を進めていきます プロダクトキーは後で入力するので\u0026quot;プロダクトキーがありません\u0026quot;を選択する \u0026ldquo;Windows 11 Pro\u0026quot;を選択する \u0026ldquo;Windows 11 をインストールする場所の選択\u0026quot;の画面になったら、上のメニューの\u0026quot;Load Driver\u0026quot;か下の\u0026quot;ハードウェアが表示されませんか？ドライバーを読み込み、ハードウェアにアクセスします。\u0026ldquo;をクリックする 画面が変わったら\u0026quot;参照\u0026quot;をクリックし、\u0026ldquo;virtio-win-0.1.226\u0026quot;を選択します \u0026ldquo;amd64\u0026rdquo;\u0026gt;\u0026ldquo;w11\u0026quot;を選択し、OKをクリックします \u0026ldquo;VirtIO SCSI pass-through controller\u0026quot;を選択して、\u0026ldquo;インストール\u0026quot;をクリックします するとディスクが表示されます 続けて他のドライバーもインストールします 上のメニューの\u0026quot;Load Driver\u0026quot;をクリックします（\u0026ldquo;適用される通知とライセンス条項\u0026quot;が表示されたら\u0026quot;同意する\u0026quot;をクリックします） \u0026ldquo;参照\u0026quot;をクリックし、\u0026ldquo;virtio-win-0.1.226\u0026quot;を選択します \u0026ldquo;NetKVM\u0026rdquo;\u0026gt;\u0026ldquo;w11\u0026rdquo;\u0026gt;\u0026ldquo;amd64\u0026quot;を選択し、OKをクリックします \u0026ldquo;VirtIO Ethernet Adapter\u0026quot;を選択して\u0026quot;インストール\u0026quot;をクリックします 同じ手順で \u0026ldquo;Balloon\u0026rdquo;\u0026gt;\u0026ldquo;w11\u0026rdquo;\u0026gt;\u0026ldquo;amd64\u0026quot;を選択し、OKをクリックします \u0026ldquo;VirtIO Balloon Driver\u0026quot;を選択して\u0026quot;インストール\u0026quot;をクリックします 次へを押します \u0026ldquo;インストール準備完了\u0026quot;が表示されたら\u0026quot;インストール\u0026quot;をクリックします インストールが始まります ここから先は通常のWindows11と同じようにセットアップしていきます ここまででWindows11のインストールは完了です\nHyper-Vのインストール Windows11のインストールが終わったらHyper-Vをインストールしていきます\nコントロールパネルを開き、\u0026ldquo;プログラム\u0026quot;をクリックします \u0026ldquo;Windowsの機能の有効化または無効化\u0026quot;をクリックし、\u0026ldquo;Hyper-V\u0026quot;にチェックを入れ、\u0026ldquo;OK\u0026quot;をクリックします\n再起動をします タスクバーのWindowsマークをクリックして\u0026quot;すべて\u0026rdquo;\u0026gt;\u0026ldquo;Windowsツール\u0026quot;をクリックします \u0026ldquo;Hyper-Vマネージャ\u0026quot;を右クリックして\u0026quot;ショートカットの作成\u0026quot;をします（デスクトップに作成しておきます） ダブルクリックで開けます qemu-guest-agentをインストールする virtio-win-0.1.266のドライブ直下に下記の2つのファイルがあるのでインストールします\nvirtio-win-gt-x64.msi virtio-win-guest-tools.exe 上記をインストールすると、Proxmoxの管理コンソールからコントロールやIPアドレスがわかるようになります ","date":"2025-03-23T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/proxmox-windows11-hyper-v-1/","title":"Proxmox上にWindows11をインストールしてHyper-V使えるようにする（1/2）"},{"content":"環境 pihole:2025.03.0 Ubuntu 24.04.2 Docker Engine 28.0.1 Docker Compose version v2.33.1 dockerとdocker-composeのインストール 1 2 3 4 5 6 7 8 9 10 11 12 13 # Add Docker\u0026#39;s official GPG key: sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources: echo \\ \u0026#34;deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \\ $(. /etc/os-release \u0026amp;\u0026amp; echo \u0026#34;${UBUNTU_CODENAME:-$VERSION_CODENAME}\u0026#34;) stable\u0026#34; | \\ sudo tee /etc/apt/sources.list.d/docker.list \u0026gt; /dev/null sudo apt-get update 1 sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin ポート53を開放する ポート53が使用されているか確認する\n1 sudo ss -lntp | grep 53 1 2 3 4 mao@pi-hole:~$ sudo ss -lntp | grep 53 LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* users:((\u0026#34;systemd-resolve\u0026#34;,pid=673,fd=15)) LISTEN 0 4096 127.0.0.54:53 0.0.0.0:* users:((\u0026#34;systemd-resolve\u0026#34;,pid=673,fd=17)) mao@pi-hole:~$ 使用しないように、下記ファイルを編集する\n1 sudo nano /etc/systemd/resolved.conf 下記項目を\u0026quot;no\u0026quot;へ変更する\n1 2 [Resolve] DNSStubListener=no リスタートする\n1 2 sudo systemctl restart systemd-resolved sudo systemctl status systemd-resolved Pi-holeを起動する フォルダを作成します\n1 mkdir pi-hole composeファイルを作成します\ncompose.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 # More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/ services: pihole: container_name: pihole #image: pihole/pihole:latest image: pihole/pihole:2025.03.0 # DHCPを機能させる場合は必須 #network_mode: host ports: # DNS Ports - \u0026#34;53:53/tcp\u0026#34; - \u0026#34;53:53/udp\u0026#34; # Default HTTP Port - \u0026#34;80:80/tcp\u0026#34; # Default HTTPs Port. FTL will generate a self-signed certificate - \u0026#34;443:443/tcp\u0026#34; # Uncomment the line below if you are using Pi-hole as your DHCP server #- \u0026#34;67:67/udp\u0026#34; # Uncomment the line below if you are using Pi-hole as your NTP server #- \u0026#34;123:123/udp\u0026#34; environment: # Set the appropriate timezone for your location (https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), e.g: #TZ: \u0026#39;Europe/London\u0026#39; TZ: \u0026#39;Asia/Tokyo\u0026#39; # Set a password to access the web interface. Not setting one will result in a random password being assigned FTLCONF_webserver_api_password: \u0026#39;maomao\u0026#39; # If using Docker\u0026#39;s default `bridge` network setting the dns listening mode should be set to \u0026#39;all\u0026#39; FTLCONF_dns_listeningMode: \u0026#39;all\u0026#39; WEBPASSWORD: maomao #WEB_PORT: 8080 # Volumes store your data between container upgrades volumes: # For persisting Pi-hole\u0026#39;s databases and common configuration file - \u0026#39;./etc-pihole:/etc/pihole\u0026#39; # Uncomment the below if you have custom dnsmasq config files that you want to persist. Not needed for most starting fresh with Pi-hole v6. If you\u0026#39;re upgrading from v5 you and have used this directory before, you should keep it enabled for the first v6 container start to allow for a complete migration. It can be removed afterwards. Needs environment variable FTLCONF_misc_etc_dnsmasq_d: \u0026#39;true\u0026#39; #- \u0026#39;./etc-dnsmasq.d:/etc/dnsmasq.d\u0026#39; cap_add: # See https://github.com/pi-hole/docker-pi-hole#note-on-capabilities # Required if you are using Pi-hole as your DHCP server, else not needed - NET_ADMIN # Required if you are using Pi-hole as your NTP client to be able to set the host\u0026#39;s system time - SYS_TIME # Optional, if Pi-hole should get some more processing time - SYS_NICE restart: unless-stopped 起動コマンドは下記の通り\n1 2 cd pi-hole sudo docker compose up -d 停止コマンドは下記の通り\n1 sudo docker compose down -v 管理画面にログインする https://192.168.10.18/admin/ ログインパスワードはcompose.yamlの\u0026quot;WEBPASSWORD:\u0026ldquo;で指定したものを入力する WebGUIの画面の右上にある\u0026quot;hostname\u0026quot;はコンテナIDになっている 1 2 3 mao@pi-hole:~$ sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7b85e233c38e pihole/pihole:2025.03.0 \u0026#34;start.sh\u0026#34; 12 days ago Up 12 days (healthy) 0.0.0.0:53-\u0026gt;53/tcp, [::]:53-\u0026gt;53/tcp, 67/udp, 0.0.0.0:80-\u0026gt;80/tcp, [::]:80-\u0026gt;80/tcp, 0.0.0.0:443-\u0026gt;443/tcp, 0.0.0.0:53-\u0026gt;53/udp, [::]:443-\u0026gt;443/tcp, [::]:53-\u0026gt;53/udp, 123/udp pihole ローカルDNSとして使う DNSレコードは\u0026quot;SYSTEM\u0026rdquo;\u0026gt;\u0026ldquo;Setting\u0026rdquo;\u0026gt;\u0026ldquo;Local DNS Records\u0026quot;に記載できる 参考URL https://github.com/pi-hole/docker-pi-hole#quick-start https://dabun.net/md/pi-hole/ ","date":"2025-03-22T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/pihole-docker-compose/","title":"Pi-holeをdocker-composeで起動する"},{"content":"環境 Harbor Version v2.11.1-6b7ecba1 Let\u0026rsquo;s Encrypt Unbound 1.19.2 Docker Engine Community 28.0.1 証明書の発行 証明書発行手順（Let\u0026#39;s Encrypt） を参照\nHarborの設定 証明書ファイルをサーバーにアップロードする\n設定ファイルを編集する\nharbor.yml 1 2 3 4 5 6 7 8 9 hostname: harbor.yourdomain.dev http: port: 80 https: port: 443 certificate: /home/mao/yourdomain.dev/cert1.pem private_key: /home/mao/yourdomain.dev/privkey1.pem 証明書はフルパスで指定する 下記を実行して設定ファイルを再生成する\n1 sudo ./install.sh 起動する\n1 sudo docker compose up -d DNS設定 DNS（Unbound）に追加する キャッシュされるまで時間がかかる？\n900s? 1 2 sudo systemctl restart unbound sudo unbound-control reload DNSに登録されているか確認する\ntraceroute dig 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 mao@mao (x86_64) : /home/mao \u0026gt; traceroute harbor.yourdomain.dev traceroute to harbor.yourdomain.dev (192.168.10.24), 30 hops max, 60 byte packets 1 192.168.10.24 (192.168.10.24) 0.403 ms 0.349 ms 0.332 ms mao@mao (x86_64) : /home/mao \u0026gt; dig harbor.yourdomain.dev ; \u0026lt;\u0026lt;\u0026gt;\u0026gt; DiG 9.18.30-0ubuntu0.24.04.2-Ubuntu \u0026lt;\u0026lt;\u0026gt;\u0026gt; harbor.yourdomain.dev ;; global options: +cmd ;; Got answer: ;; -\u0026gt;\u0026gt;HEADER\u0026lt;\u0026lt;- opcode: QUERY, status: NOERROR, id: 62907 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ;harbor.yourdomain.dev.\tIN\tA ;; ANSWER SECTION: harbor.yourdomain.dev. 3123 IN\tA\t192.168.10.24 ;; Query time: 0 msec ;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP) ;; WHEN: Sat Mar 08 17:00:06 JST 2025 ;; MSG SIZE rcvd: 72 Harborからpullできるか確認する pullを実行するとエラーになった\n1 2 3 mao@mao (x86_64) : /home/mao \u0026gt; sudo docker pull harbor.yourdomain.dev/homelab/homelab-traefik:3.1.6 Error response from daemon: Get \u0026#34;https://harbor.yourdomain.dev/v2/\u0026#34;: tls: failed to verify certificate: x509: certificate signed by unknown authority エラーの対処法 証明書（公開鍵）と中間証明書を結合したものが必要なよう\nhttps://goharbor.io/docs/2.0.0/install-config/troubleshoot-installation/#https 一度止める\n1 sudo docker compose down -v 設定ファイルを再度編集する\nharbor.yml 1 2 3 4 5 6 7 8 9 10 hostname: harbor.yourdomain.dev http: port: 80 https: port: 443 - certificate: /home/mao/yourdomain.dev/cert1.pem + certificate: /home/mao/yourdomain.dev/fullchain1.pem private_key: /home/mao/yourdomain.dev/privkey1.pem 証明書と中間証明書が結合された\u0026quot;fullchain1.pem\u0026quot;を使用するように編集する \u0026ldquo;certificate: /home/mao/yourdomain.dev/cert1.pem\u0026quot;ではなく \u0026ldquo;certificate: /home/mao/yourdomain.dev/fullchain1.pem\u0026quot;とする 下記を実行して設定ファイルを再生成する\n1 sudo ./install.sh 起動する\n1 sudo docker compose up -d 改めてpullしてみる 下記コマンドでpullをする\n1 sudo docker pull harbor.yourdomain.dev/homelab/homelab-traefik:3.1.6 ちゃんとpullされたことが確認できた\n1 2 3 4 5 6 7 8 9 10 mao@mao (x86_64) : /home/mao \u0026gt; sudo docker pull harbor.yourdomain.dev/homelab/homelab-traefik:3.1.6 3.1.6: Pulling from homelab/homelab-traefik 43c4264eed91: Pull complete f60fb4c0fbec: Pull complete 9a6d31097c4f: Pull complete e5f06ee63d76: Pull complete Digest: sha256:22aec04848987fe5b3999a4099d766de614b04da52a936fc5ac214ffec04dbac Status: Downloaded newer image for harbor.yourdomain.dev/homelab/homelab-traefik:3.1.6 harbor.yourdomain.dev/homelab/homelab-traefik:3.1.6 これでHorborからhttpsでpullができたので、daemon.jsonファイルを編集しなくても使えるようになった\n参考URL OSSのコンテナレジストリ「Harbor」を自己署名証明書でHTTPS通信させる https://tech-mmmm.blogspot.com/2023/05/ossharborhttps.html HTTPS接続のトラブルシューティング https://goharbor.io/docs/2.0.0/install-config/troubleshoot-installation/#https https://github.com/goharbor/harbor/issues/6774 Let\u0026rsquo;s Encrypt(無料SSL証明書)についてまとめ https://qiita.com/morrr/items/4d96d7a52c7a54e22bf3 ","date":"2025-03-14T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/harbor-https-docker-pull/","title":"Harborにhttpsでアクセスしてdocker pullできるようにする"},{"content":"理由 ローカル内にあるDNSで名前解決して、そのDNSが落ちているときは別のDNSで通信自体はできるようにしたかった\nそこでセカンダリDNSをどのように追加したら良いのか調べた\nUbuntuDesktopのGUIの場合 \u0026ldquo;Settings\u0026rdquo;\u0026gt;\u0026ldquo;Network\u0026rdquo;\u0026gt;\u0026ldquo;Ethernet(eno1)\u0026quot;\u0026gt;\u0026ldquo;IPv4\u0026rdquo;\u0026gt;\u0026ldquo;DNS\u0026rdquo; \u0026ldquo;192.168.10.1, 192.168.10.109\u0026rdquo; カンマ区切りで入れられる 先にあるのがプライマリで、その次がセカンダリになる UbuntuServerのCUI（Netplanファイル）の場合 追加前 1 2 3 4 5 6 7 8 9 10 11 network: version: 2 ethernets: eno1: renderer: NetworkManager match: {} addresses: - \u0026#34;192.168.10.50/24\u0026#34; nameservers: addresses: - 192.168.10.1 追加後 1 2 3 4 5 6 7 8 9 10 11 12 network: version: 2 ethernets: eno1: renderer: NetworkManager match: {} addresses: - \u0026#34;192.168.10.50/24\u0026#34; nameservers: addresses: - 192.168.10.1 - 192.168.10.109 \u0026ldquo;resolvectl\u0026quot;コマンドで確認する 追加前 1 2 3 4 5 6 7 8 9 10 11 mao@mao (x86_64) : /home/mao \u0026gt; resolvectl Global Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported resolv.conf mode: stub Link 2 (eno1) Current Scopes: DNS Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported Current DNS Server: 192.168.10.1 DNS Servers: 192.168.10.1 追加後 1 2 3 4 5 6 7 8 9 10 11 mao@mao (x86_64) : /home/mao \u0026gt; resolvectl Global Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported resolv.conf mode: stub Link 2 (eno1) Current Scopes: DNS Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported Current DNS Server: 192.168.10.1 DNS Servers: 192.168.10.1 192.168.10.109 \u0026ldquo;DNS Servers\u0026quot;の項目に追加されている\n運用 プライマリをローカルのDNSにすれば、ローカル内の名前解決を先にしてないものはパブリックのDNSに問い合わせる形にできる かつ、ローカルのDNSがダウンしていてもセカンダリのDNSに問い合わせる形にできる\n","date":"2025-03-13T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/secondary-dns/","title":"セカンダリDNSの追加"},{"content":"理由 2025年4月1日から認証されていないユーザーは、1時間あたり10pullに制限されるみたいなので回避策を試してみる\nhttps://docs.docker.com/docker-hub/usage/ 環境 Ubuntu 24.04.2 LTS Docker Engine Community Version 28.0.1 設定ファイルを作成する デフォルトでは設定ファイルがないのでファイルを作成する\n\u0026ldquo;etc/docker/\u0026ldquo;に\u0026quot;daemon.json\u0026quot;ファイルがあるか確認し、ない場合は下記の手順で設定ファイルを作成する 1 ls /etc/docker \u0026ldquo;daemon.json\u0026quot;ファイルを作成する\n1 2 touch daemon.json nano deamon.json 下記を追記する（参考URL）\nhttps://cloud.google.com/artifact-registry/docs/pull-cached-dockerhub-images?hl=ja 1 2 3 { \u0026#34;registry-mirrors\u0026#34;: [\u0026#34;https://mirror.gcr.io\u0026#34;] } 他の設定もある場合は下記のように\u0026rdquo;,\u0026ldquo;を入れる\n例 1 2 3 4 { \u0026#34;insecure-registries\u0026#34;: [\u0026#34;192.168.10.17\u0026#34;], \u0026#34;registry-mirrors\u0026#34;: [\u0026#34;https://mirror.gcr.io\u0026#34;] } 設定ファイルをコピーしてdockerを再起動する\n1 2 sudo cp -f ./daemon.json /etc/docker/daemon.json sudo systemctl restart docker 次回以降編集する場合または、すでに設定ファイルがある場合は、下記のように設定ファイルを編集してdockerを再起動する\n1 2 sudo nano /etc/docker/daemon.json sudo systemctl restart docker 設定が反映されているか確認する 下記コマンドで確認する\n1 sudo docker system info \u0026ldquo;Registry Mirrors:\u0026ldquo;の項目に追加されている 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 mao@mao (x86_64) : /home/mao \u0026gt; sudo docker system info Client: Docker Engine - Community Version: 28.0.1 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.21.1 Path: /usr/libexec/docker/cli-plugins/docker-buildx Server: Containers: 19 Running: 0 Paused: 0 Stopped: 19 Images: 6 Server Version: 28.0.1 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb runc version: v1.2.4-0-g6c52b3f init version: de40ad0 Security Options: apparmor seccomp Profile: builtin cgroupns Kernel Version: 6.8.0-55-generic Operating System: Ubuntu 24.04.2 LTS OSType: linux Architecture: x86_64 CPUs: 20 Total Memory: 62.54GiB Name: mao ID: ****** Docker Root Dir: /var/lib/docker Debug Mode: false Experimental: false Insecure Registries: 192.168.10.17 ::1/128 127.0.0.0/8 Registry Mirrors: https://mirror.gcr.io/ Live Restore Enabled: false 参考URL DockerHubのレート制限を受けないようにmirror.gcr.ioを使う https://blog.monophile.net/posts/20201101_docker_mirror_gcr_io.html キャッシュに保存された Docker Hub イメージの pull https://cloud.google.com/artifact-registry/docs/pull-cached-dockerhub-images?hl=ja daemon.jsonでDockerエンジンのオプション指定 (/etc/docker, insecure-registries) https://devlights.hatenablog.com/entry/2021/11/30/010150 dockerd daemon https://docs.docker.com/reference/cli/dockerd/#daemon-configuration-file ","date":"2025-03-08T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/dockerhub-mirror-gcr/","title":"DockerHubのミラーを設定する"},{"content":"環境 Raspberry Pi 5 Model B 8GB（この記事では\u0026quot;ラズパイ5\u0026quot;と記載） KIOXIA SATA SSD 480GB Ubuntu 24.04.1 LTS aarch64 USB-SATA変換アダプタ Raspberry Pi Imager v1.8.5 ラズパイ5を起動する準備 \u0026ldquo;Raspberry Pi Imager\u0026quot;を使用して、SSDにOSイメージを書き込みます 書き込みが完了\nラズパイ5にUSB-SSDとLANケーブル、電源ケーブルを接続します\n電源を入れて30秒程たったら電源ボタンを押します（電源容量不足の警告が出るため）\nすると、bootが始まります\n別のPCからsshでログインします\nSSHで接続する 1 ssh mao@raspberrypi5.local パスワードが求められるので入力する IPアドレスがわからなくても接続できる 必要なソフトウェアをインストール 1 2 sudo apt install neofetch sudo apt install nano 電源容量不足の警告が出ないようにする \u0026ldquo;/boot/firmware/config.txt\u0026quot;に\u0026quot;usb_max_current_enable=1\u0026quot;を追加する\n1 sudo nano /boot/firmware/config.txt neofetch 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 mao@raspberrypi5:~$ neofetch .-/+oossssoo+/-. mao@raspberrypi5 `:+ssssssssssssssssss+:` ---------------- -+ssssssssssssssssssyyssss+- OS: Ubuntu 24.04.1 LTS aarch64 .ossssssssssssssssssdMMMNysssso. Host: Raspberry Pi 5 Model B Rev 1.0 /ssssssssssshdmmNNmmyNMMMMhssssss/ Kernel: 6.8.0-1010-raspi +ssssssssshmydMMMMMMMNddddyssssssss+ Uptime: 5 mins /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/ Packages: 739 (dpkg), 2 (snap) .ssssssssdMMMNhsssssssssshNMMMdssssssss. Shell: bash 5.2.21 +sssshhhyNMMNyssssssssssssyNMMMysssssss+ Terminal: /dev/pts/0 ossyNMMMNyMMhsssssssssssssshmmmhssssssso CPU: (4) @ 2.400GHz ossyNMMMNyMMhsssssssssssssshmmmhssssssso Memory: 183MiB / 7937MiB +sssshhhyNMMNyssssssssssssyNMMMysssssss+ .ssssssssdMMMNhsssssssssshNMMMdssssssss. /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/ +sssssssssdmydMMMMMMMMddddyssssssss+ /ssssssssssshdmNNNNmyNMMMMhssssss/ .ossssssssssssssssssdMMMNysssso. -+sssssssssssssssssyyyssss+- `:+ssssssssssssssssss+:` .-/+oossssoo+/-. mao@raspberrypi5:~$ ディスク容量の確認 下記コマンドを実行する\n1 2 sudo fdisk -l df -h --total 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 mao@raspberrypi5:~$ sudo fdisk -l Disk /dev/loop0: 33.71 MiB, 35344384 bytes, 69032 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/loop1: 38.57 MiB, 40443904 bytes, 78992 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/sda: 447.13 GiB, 480103981056 bytes, 937703088 sectors Disk model: XCERIA SATA SSD Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 33553920 bytes Disklabel type: dos Disk identifier: 0x9b94b926 Device Boot Start End Sectors Size Id Type /dev/sda1 * 2048 1050623 1048576 512M c W95 FAT32 (LBA) /dev/sda2 1050624 937703054 936652431 446.6G 83 Linux 1 2 3 4 5 6 7 8 9 10 mao@raspberrypi5:~$ df -h --total Filesystem Size Used Avail Use% Mounted on tmpfs 794M 3.2M 791M 1% /run /dev/sda2 440G 2.2G 420G 1% / tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda1 505M 91M 414M 18% /boot/firmware tmpfs 794M 12K 794M 1% /run/user/1000 total 446G 2.3G 426G 1% - mao@raspberrypi5:~$ シャットダウン 1 sudo systemctl poweroff ","date":"2025-03-02T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/raspberrypi5-usb-boot/","title":"ラズパイ5をUSBのSSDで起動する（USBブート,USB-Boot）"},{"content":"なんのために？ 家庭内で普段使っているサブネットと、サーバー等のサブネットを分けてようと思ったため。\nしかし、そのままだと家庭内で使用しているサブネットからサーバーにアクセスできないので、サブネットを越えて通信させたかったので設定方法を調べた結果、スタティックルーティングという方法でルーティングさせて通信させることにした。\n環境 Ubuntu Server 24.04.1 ネットワーク図 家庭内のサブネット：192.168.10.0/24 サーバー等のサブネット：192.168.5.0/24 IPv4のフォワードを有効にする ネットワーク図上の\u0026quot;VM2\u0026quot;のIPv4のフォワードを有効にする 有効になっているか設定を確認する\n1 cat /proc/sys/net/ipv4/ip_forward 結果\n\u0026ldquo;0\u0026rdquo;：IPフォワーディングが無効になっている \u0026ldquo;1\u0026rdquo;：IPフォワーディングが有効になっている 一時的に有効化する 再起動をするとリセットされる\n1 echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 永続的に有効化する 再起動をしてもリセットされないようにする\n下記の設定ファイルを編集する\n1 sudo nano /etc/sysctl.conf 下記を追加する\n1 net.ipv4.ip_forward=1 設定を反映する\n1 sudo sysctl -p ルーターにするVM2に設定する \u0026ldquo;VM2\u0026quot;についているNIC（ens18/ens19）の固定IPアドレスを設定する 設定ファイルをコピーする\n1 sudo cp /etc/netplan/50-cloud-init.yaml /etc/netplan/99-config.yaml 設定ファイルを編集する\n1 sudo nano /etc/netplan/99-config.yaml 設定を反映させる\n1 sudo netplan apply 99-config.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 network: version: 2 renderer: networkd ethernets: ens18: dhcp4: false addresses: - 192.168.5.110/24 #routes: #- to: 192.168.5.0/24 #scope: link ens19: dhcp4: false addresses: - 192.168.10.110/24 #routes: #- to: 192.168.10.0/24 #scope: link スタティックルートを設定する 直接接続のためルーティング設定をしなくても自動的に設定される\n設定されているか確認をする\n1 ip route 1 2 192.168.5.0/24 dev ens18 proto kernel scope link src 192.168.10.110 192.168.10.0/24 dev ens19 proto kernel scope link src 192.168.5.110 それぞれのサブネットにあるクライアントにスタティックルートを設定する ※OSはどちらもUbuntuServer24.04.1\n一時的に追加する \u0026ldquo;ip route add\u0026quot;コマンドで追加したスタティックルートは再起動をすると消えてしまう\n\u0026ldquo;192.168.10.0/24\u0026quot;にある\u0026quot;VM1(192.168.10.50)\u0026ldquo;のPCに設定\n1 sudo ip route add 192.168.5.0/24 via 192.168.10.35 dev eno1 記載例：\u0026ldquo;ip route add \u0026lt;対象へのネットワーク\u0026gt; via \u0026lt;ゲートウェイ\u0026gt; dev \u0026lt;インターフェース名\u0026gt;\u0026rdquo; \u0026ldquo;192.168.5.0/24\u0026quot;にある\u0026quot;VM3(192.168.5.108)\u0026ldquo;のPCに設定\n1 sudo ip route add 192.168.10.0/24 via 192.168.5.110 dev ens18 永続的に追加する netplanの設定ファイルにスタティックルートを追加する\n1 2 3 4 5 6 7 8 9 10 11 network: version: 2 renderer: networkd ethernets: ens18: dhcp4: false addresses: - 192.168.10.50/24 routes: - to: 192.168.5.0/24 via: 192.168.10.110 下記は\u0026quot;192.168.5.0/24\u0026quot;のサブネットへの通信をゲートウェイ\u0026quot;192.168.10.110\u0026quot;へ送るスタティックルートの設定の例\n1 2 3 routes: - to: 192.168.5.0/24 via: 192.168.10.110 設定を反映させる\n1 sudo netplan apply ルーティングが設定されているか確認する\n1 ip route 疎通しているかpingで確認する pingを飛ばして確認する\n\u0026ldquo;VM1(192.168.10.50)\u0026ldquo;から下記コマンドを実行する 1 ping 192.168.5.108 実行結果\n1 2 3 4 5 6 7 8 9 10 ping 192.168.5.108 PING 192.168.5.108 (192.168.5.108) 56(84) bytes of data. 64 bytes from 192.168.5.108: icmp_seq=1 ttl=63 time=0.789 ms 64 bytes from 192.168.5.108: icmp_seq=2 ttl=63 time=0.991 ms 64 bytes from 192.168.5.108: icmp_seq=3 ttl=63 time=0.920 ms 64 bytes from 192.168.5.108: icmp_seq=4 ttl=63 time=0.933 ms ^C --- 192.168.5.108 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3041ms rtt min/avg/max/mdev = 0.789/0.908/0.991/0.073 ms 問題なければ、サブネットを越えて通信ができた\nネットワーク図 参考URL Netplan documentation https://netplan.readthedocs.io/en/latest/netplan-yaml/#default-routes 【Ubuntu18.04のネットワーク設定方法】Netplanの解説 https://www.komee.org/entry/2018/06/13/170000 Ubuntu netplan書き方の基本設定とトラブル対処法 https://hamagen.blog/ubuntu-netplan-write/ サーバーをルーターにする設定「/proc/sys/net/ipv4/ip_forward」とは? https://note.com/minato_kame/n/n8002de5a9b03 VLAN の中で IP Forwarding を設定する(Akamai) https://qiita.com/110hideki/items/851291f0d33acc577a82#ip-forwarding-%E3%81%AE%E8%A8%AD%E5%AE%9A ルーティングテーブルを表示・操作するコマンドを確認する https://takapi86.hatenablog.com/entry/2019/09/23/232240 【Linux】ip routeで表示されるdev,via,src,protoの意味と見方 https://www.mtioutput.com/entry/iproute-cmd-howtosee 基本的なTCP/IP通信の仕組み（別セグメント間の通信） https://infrastructure-engineer.com/tcpip-basic-0002/ 【Linux】静的ルーティングをコマンドで追加・削除・確認する方法 https://infra-exp.com/linux_static_route/ ","date":"2025-01-02T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/static-routing/","title":"スタティックルーティング（Static-Routing、静的ルーティング）を設定して別のサブネットと通信できるようにする"},{"content":"2024年振り返り 今年の目標としていた、基本情報と応用情報の合格は両方受験をして、基本情報技術者のみ合格することができました。\n基本情報に合格できた理由は、先に応用を受験して落ちていたので、その反省から、勉強時間を増やして試験に望んだからだと思っています。\n勉強の仕方もただ覚えるのではなく「どのような仕組みなのか」を意識しながら勉強したことも良かったのかなと思っています。\n応用情報に合格できなかった点としては、難易度を軽く見ていたことによる勉強不足と思っています。\nなので、来年受験するときには前回受けたときより知識も増えてはいると思いますが、もっと勉強時間を増やし2025年に合格することを目指します。\n他にはkubernetesの勉強がてら1から構築してCNIを色々試してみたり、ブログサイトをHugoで作って記事を31個作成したりしました。\n来年はkubernetes上で動くアプリケーションを色々試したりGoで作成したり、電子工作もしているので技術を融合させて作ってみようと思っています。\nブログに関しては来年はもっと色々と挑戦して勉強して、結果として今年よりも多く記事を書けたらと思っています。\n2025年の目標 来年2025年の目標は「応用情報技術者試験合格」、「LPIC1合格」、「1つ山に登る」、「自分で基板から作ったIoT関係のものを作る」、を頑張っていきます。\n","date":"2024-12-31T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/2024review-and-2025plan/","title":"2024年振り返りと2025年の目標"},{"content":"背景 ローカルの開発環境をHTTPSでアクセスしたいが、自己署名証明書だと警告が出たりと問題があるので、正規の証明書を発行して使用する\n前回の記事で証明書を発行したので、今回はnginxに証明書をコピーし、HTTPSでアクセスできるようにする\n環境 2VM\nUnbound(DNS) unbound 1.19.2. Nginx nginx version: nginx/1.24.0 (Ubuntu) nginxのインストールと起動 インストールと状態の確認をする\n1 2 3 4 sudo apt install nano sudo apt install nginx sudo systemctl start nginx sudo systemctl status nginx \u0026ldquo;Active\u0026quot;が\u0026quot;active (running)\u0026ldquo;になっていればOK\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 mao@internal-domain:~$ sudo systemctl status nginx ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: enabled) Active: active (running) since Sun 2024-12-15 01:33:08 UTC; 46s ago Docs: man:nginx(8) Process: 17202 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS) Process: 17204 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS) Main PID: 17205 (nginx) Tasks: 5 (limit: 9445) Memory: 3.6M (peak: 3.9M) CPU: 9ms CGroup: /system.slice/nginx.service ├─17205 \u0026#34;nginx: master process /usr/sbin/nginx -g daemon on; master_process on;\u0026#34; ├─17206 \u0026#34;nginx: worker process\u0026#34; ├─17207 \u0026#34;nginx: worker process\u0026#34; ├─17208 \u0026#34;nginx: worker process\u0026#34; └─17209 \u0026#34;nginx: worker process\u0026#34; Dec 15 01:33:08 internal-domain systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server... Dec 15 01:33:08 internal-domain systemd[1]: Started nginx.service - A high performance web server and a reverse proxy server. IPアドレスにアクセスしてnginxのスタートページが表示されることを確認する\n証明書を入れるフォルダを作成し、証明書ファイルをアップロードする 証明書を入れるフォルダを作成する\n1 sudo mkdir /etc/nginx/ssl ローカルのPCからNginxがインストールされているサーバーへ証明書ファイルをアップロードする\n1 scp -r /home/mao/internal.onodera-program.com mao@192.168.10.30:/home/mao/ 1 2 3 4 5 6 7 8 mao@mao (x86_64) : /home/mao \u0026gt; scp -r /home/mao/internal.onodera-program.com mao@192.168.10.30:/home/mao/ mao@192.168.10.30\u0026#39;s password: cert1.pem 100% 1306 927.0KB/s 00:00 fullchain1.pem 100% 2872 2.4MB/s 00:00 privkey1.pem 100% 241 529.7KB/s 00:00 README 100% 692 1.8MB/s 00:00 chain1.pem 100% 1566 3.7MB/s 00:00 Nginxの設定ファイルを作成する 設定ファイルを作成し、編集します\n1 sudo nano /etc/nginx/conf.d/default.conf default.conf 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 server { listen 80; listen 443 ssl; listen [::]:80; listen [::]:443 ssl; server_name localhost; server_name internal.onodera-program.com; ssl_certificate /etc/nginx/ssl/fullchain1.pem; ssl_certificate_key /etc/nginx/ssl/privkey1.pem; location / { root /usr/share/nginx/html; index index.html index.htm; } #error_page 404 /404.html; # redirect server error pages to the static page /50x.html # error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } } \u0026ldquo;server_name\u0026quot;は使用するドメインを記載する \u0026ldquo;ssl_certificate\u0026quot;と\u0026quot;ssl_certificate_key\u0026quot;はアップロードしたファイルをフルパスで記載する 設定を読み込むためにnginxを再起動する\n1 2 sudo systemctl restart nginx sudo systemctl status nginx DNSサーバーの設定をする ローカル内にあるDNSサーバー（今回はUnboundを構築した）に、IPアドレスとドメイン名を設定する\nアクセスするPCのDNSを上記設定がされているDNSサーバーのIPアドレスを指定する\nドメイン名＋HTTPSでアクセスしてみる \u0026ldquo;https://ドメイン名\u0026quot;にアクセスする\n例：\u0026ldquo;https://internal.onodera-program.com\u0026rdquo; 証明書の警告等が表示されずにアクセスできればOK その他 あとは同じ要領で証明書を他のサーバーに登録して、ローカルのサーバーにHTTPSでアクセスできるようにする\nk8sのIngressに登録すれば、ローカル内でHTTPS＋ドメイン名でアクセスできるようになる、はず\n参考URL Let\u0026rsquo;s Encrypt で Nginx にSSLを設定する https://qiita.com/HeRo/items/f9eb8d8a08d4d5b63ee9 Nginxに自己署名証明書を設定してHTTPS接続してみる https://qiita.com/ohakutsu/items/814825a76b5299a96661 Linuxでリモート・ローカル間でファイルを転送するコマンド https://uxmilk.jp/50946 127.0.0.1にLet\u0026rsquo; Encryptで証明書を発行してhttpsでのローカル開発と本番の差異を低減するための具体的な手順 https://scrapbox.io/nwtgck/127.0.0.1%E3%81%ABLet'_Encrypt%E3%81%A7%E8%A8%BC%E6%98%8E%E6%9B%B8%E3%82%92%E7%99%BA%E8%A1%8C%E3%81%97%E3%81%A6https%E3%81%A7%E3%81%AE%E3%83%AD%E3%83%BC%E3%82%AB%E3%83%AB%E9%96%8B%E7%99%BA%E3%81%A8%E6%9C%AC%E7%95%AA%E3%81%AE%E5%B7%AE%E7%95%B0%E3%82%92%E4%BD%8E%E6%B8%9B%E3%81%99%E3%82%8B%E3%81%9F%E3%82%81%E3%81%AE%E5%85%B7%E4%BD%93%E7%9A%84%E3%81%AA%E6%89%8B%E9%A0%86 ローカルホスト（127.0.0.1）用のSSL証明書を取得する https://hiroshi-nagayama.com/%E3%83%AD%E3%83%BC%E3%82%AB%E3%83%AB%E3%83%9B%E3%82%B9%E3%83%88%EF%BC%88127-0-0-1%EF%BC%89%E7%94%A8%E3%81%AEssl%E8%A8%BC%E6%98%8E%E6%9B%B8%E3%82%92%E5%8F%96%E5%BE%97%E3%81%99%E3%82%8B/ ローカル開発環境の https 化 https://blog.jxck.io/entries/2020-06-29/https-for-localhost.html ","date":"2024-12-18T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/internal-https-domain/","title":"ローカル内で自己署名証明書を使わずにドメイン名＋HTTPSでアクセスする"},{"content":"背景 ローカルの開発環境をHTTPSでアクセスしたいが、自己署名証明書だと警告が出たりと問題があるので、正規の証明書を発行して使用する\n証明書の発行準備 証明書を発行する用のVMを作ったのでsshで接続する\n1 ssh mao@192.168.10.11 certbotをインストールする\n1 sudo apt install certbot オプションを確認するためにヘルプを参照する\n1 certbot help all 下記は今回必要なオプションを抜粋して記載（実際は全て表示される）\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 mao@lets-encrypt-certbot:~$ certbot help all usage: certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ... Certbot can obtain and install HTTPS/TLS/SSL certificates. By default, it will attempt to use a webserver both for obtaining and installing the certificate. The most common SUBCOMMANDS and flags are: obtain, install, and renew certificates: certonly Obtain or renew a certificate, but do not install it -d DOMAINS Comma-separated list of domains to obtain a certificate for --manual Obtain certificates interactively, or using shell script hooks options: -h, --help show this help message and exit -d DOMAIN, --domains DOMAIN, --domain DOMAIN Domain names to include. For multiple domains you can use multiple -d flags or enter a comma separated list of domains as a parameter. All domains will be included as Subject Alternative Names on the certificate. The first domain will be used as the certificate name, unless otherwise specified or if you already have a certificate with the same name. In the case of a name conflict, a number like -0001 will be appended to the certificate name. (default: Ask) register: Options for account registration --register-unsafely-without-email Specifying this flag enables registering an account with no email address. This is strongly discouraged, because you will be unable to receive notice about impending expiration or revocation of your certificates or problems with your Certbot installation that will lead to failure to renew. (default: False) -m EMAIL, --email EMAIL Email used for registration and recovery contact. Use comma to register multiple emails, ex: u1@example.com,u2@example.com. (default: Ask). SSL証明書を発行する 実行するコマンドは下記の通り\n1 2 3 4 5 6 certbot certonly --manual --server https://acme-v02.api.letsencrypt.org/directory --preferred-challenges dns -d internal.onodera.dev --register-unsafely-without-email オプションの説明（発行するドメインに置き換える）\n\u0026ldquo;certonly\u0026rdquo; 証明書の発行のみをする \u0026ldquo;\u0026ndash;manual\u0026rdquo; マニュアルで指定する \u0026ldquo;\u0026ndash;server https://acme-v02.api.letsencrypt.org/directory\" 証明書の発行をするサーバーを指定する \u0026ldquo;\u0026ndash;preferred-challenges dns\u0026rdquo; チャレンジ方法をDNSにする \u0026ldquo;-d *.internal.onodera.dev\u0026rdquo; 証明書を発行するドメインを指定する 今回はサブドメインで証明書を発行する \u0026ldquo;\u0026ndash;register-unsafely-without-email\u0026rdquo; メールアドレスを登録せずに証明書を発行する コマンド実行 1 sudo certbot certonly --manual --server https://acme-v02.api.letsencrypt.org/directory --preferred-challenges dns -d internal.onodera-program.com --register-unsafely-without-email 実行結果 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 mao@lets-encrypt-certbot:~$ sudo certbot certonly --manual --server https://acme-v02.api.letsencrypt.org/directory --preferred-challenges dns -d internal.onodera-program.com --register-unsafely-without-email Saving debug log to /var/log/letsencrypt/letsencrypt.log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Please read the Terms of Service at https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf. You must agree in order to register with the ACME server. Do you agree? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - (Y)es/(N)o: Y Account registered. Requesting a certificate for internal.onodera-program.com - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Please deploy a DNS TXT record under the name: _acme-challenge.internal.onodera-program.com. with the following value: ***1b5eLnL15fqIdYkuSaqRu1CPGmgq6FTgnEHbZVTs Before continuing, verify the TXT record has been deployed. Depending on the DNS provider, this may take some time, from a few seconds to multiple minutes. You can check if it has finished deploying with aid of online tools, such as the Google Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.internal.onodera-program.com. Look for one or more bolded line(s) below the line \u0026#39;;ANSWER\u0026#39;. It should show the value(s) you\u0026#39;ve just added. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Press Enter to Continue Successfully received certificate. Certificate is saved at: /etc/letsencrypt/live/internal.onodera-program.com/fullchain.pem Key is saved at: /etc/letsencrypt/live/internal.onodera-program.com/privkey.pem This certificate expires on 2025-03-08. These files will be updated when the certificate renews. NEXT STEPS: - This certificate will not be renewed automatically. Autorenewal of --manual certificates requires the use of an authentication hook script (--manual-auth-hook) but one was not provided. To renew this certificate, repeat this same certbot command before the certificate\u0026#39;s expiry date. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - If you like Certbot, please consider supporting our work by: * Donating to ISRG / Let\u0026#39;s Encrypt: https://letsencrypt.org/donate * Donating to EFF: https://eff.org/donate-le - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - mao@lets-encrypt-certbot:~$ 発行手順 利用規約を読み同意するか求められるので、\u0026ldquo;Y\u0026quot;を押す DNSのTXTレコードに値を登録するよう求められるので、DNSに値を登録する DNSにTXTレコードが反映されているかdigコマンド等で確認する TXTレコードが反映されていたら、Enterを押す 証明書が発行される TXTレコードの確認方法 下記コマンドを実行する（ドメイン名は登録したドメインを指定）\n1 dig -t txt _acme-challenge.internal.onodera-program.com \u0026ldquo;ANSWER SECTION:\u0026ldquo;に値が記載されていれば反映されている\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 ; \u0026lt;\u0026lt;\u0026gt;\u0026gt; DiG 9.10.6 \u0026lt;\u0026lt;\u0026gt;\u0026gt; -t txt _acme-challenge.internal.onodera-program.com ;; global options: +cmd ;; Got answer: ;; -\u0026gt;\u0026gt;HEADER\u0026lt;\u0026lt;- opcode: QUERY, status: NOERROR, id: 61386 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ;; QUESTION SECTION: ;_acme-challenge.internal.onodera-program.com. IN TXT ;; ANSWER SECTION: _acme-challenge.internal.onodera-program.com. 300 IN TXT \u0026#34;***1b5eLnL15fqIdYkuSaqRu1CPGmgq6FTgnEHbZVTs\u0026#34; ;; Query time: 93 msec ;; SERVER: 192.168.10.1#53(192.168.10.1) ;; WHEN: Sun Dec 08 11:32:45 JST 2024 ;; MSG SIZE rcvd: 129 証明書をダウンロードする 所有者変更 chown:\u0026ldquo;change owner\u0026rdquo;:所有者変更 chmod:\u0026ldquo;change mode\u0026rdquo;:アクセス権変更 注意1\n下記の方法でコピーしてもファイルの実態は別の場所にあり、シンボリックリンクがコピーされるだけなので、ローカルにダウンロードする際に権限エラーになってしまうので、注意2の方法でダウンロードできる\n1 2 3 4 5 sudo su - cp -r /etc/letsencrypt/live/internal.onodera-program.com /home/mao cd /home/mao chown -R mao internal.onodera-program.com exit 1 2 3 4 5 6 7 8 9 10 11 12 13 mao@lets-encrypt-certbot:~$ sudo su - root@lets-encrypt-certbot:~# cp -r /etc/letsencrypt/live/internal.onodera-program.com /home/mao root@lets-encrypt-certbot:~# cd /home/mao root@lets-encrypt-certbot:/home/mao# ls -l total 4 drwxr-xr-x 2 root root 4096 Dec 8 03:05 internal.onodera-program.com root@lets-encrypt-certbot:/home/mao# chown -R mao internal.onodera-program.com root@lets-encrypt-certbot:/home/mao# ls -l total 4 drwxr-xr-x 2 mao root 4096 Dec 8 03:05 internal.onodera-program.com root@lets-encrypt-certbot:/home/mao# exit logout mao@lets-encrypt-certbot:~$ 注意2\n上記のファイルをコピーしても別の場所にあるファイルをリンクしているだけなのでダウンロード時にエラーになってしまったので、 ファイル本体をコピーして所有権を変更する\nシンボリックリンクのファイル本体（リンク先）がどこにあるか確認する\n1 2 3 4 5 6 7 mao@lets-encrypt-certbot:~/internal.onodera-program.com$ ls -l total 4 -rw-r--r-- 1 mao root 692 Dec 8 03:05 README lrwxrwxrwx 1 mao root 52 Dec 8 03:05 cert.pem -\u0026gt; ../../archive/internal.onodera-program.com/cert1.pem lrwxrwxrwx 1 mao root 53 Dec 8 03:05 chain.pem -\u0026gt; ../../archive/internal.onodera-program.com/chain1.pem lrwxrwxrwx 1 mao root 57 Dec 8 03:05 fullchain.pem -\u0026gt; ../../archive/internal.onodera-program.com/fullchain1.pem lrwxrwxrwx 1 mao root 55 Dec 8 03:05 privkey.pem -\u0026gt; ../../archive/internal.onodera-program.com/privkey1.pem 変更手順 root権限にする\n1 sudo su - ファイルをホームディレクトリへコピーする\n1 cp -r /etc/letsencrypt/archive/internal.onodera-program.com /home/mao 所有者を変更する\n1 2 cd /home/mao chown -R mao internal.onodera-program.com root権限から抜ける\n1 exit 証明書をローカルにダウンロードする Macで作業 1 scp -r mao@192.168.10.11:/home/mao/internal.onodera-program.com /Users/username 1 2 3 4 5 6 7 % \u0026gt; scp -r mao@192.168.10.11:/home/mao/internal.onodera-program.com /Users/username mao@192.168.10.11\u0026#39;s password: privkey1.pem 100% 241 35.4KB/s 00:00 README 100% 692 57.3KB/s 00:00 fullchain1.pem 100% 2872 272.6KB/s 00:00 cert1.pem 100% 1306 134.0KB/s 00:00 chain1.pem ダウンロードしたファイルの説明\n\u0026ldquo;privkey1.pem\u0026rdquo;:秘密鍵 \u0026ldquo;fullchain1.pem\u0026rdquo;:証明書＋中間証明書の連結ファイル \u0026ldquo;cert1.pem\u0026rdquo;:証明書 \u0026ldquo;chain1.pem\u0026rdquo;:中間証明書 この証明書をアップロードしたりして使用できる\n参考URL Certbot documentation https://eff-certbot.readthedocs.io/en/stable/ Let’s Encrypt ドキュメント https://letsencrypt.org/ja/docs/ 無料SSL証明書のLet’s Encryptとは？ https://ssl.sakura.ad.jp/column/letsencrypt/ シンボリックリンクの作成と削除 https://qiita.com/colorrabbit/items/2e99304bd92201261c60 ","date":"2024-12-17T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/certbot-lets-encrypt/","title":"証明書発行手順（Let's Encrypt）"},{"content":"背景 複数のPCからaptを使用する際に毎回ダウンロードすると時間がかかるので、ローカルにキャッシュサーバーを立ててそこからダウンロードできるようにしたい\nキャッシュサーバーではなくミラーサーバーでも良かったが、ミラーサーバーだと容量が必要になるので、今回はキャッシュサーバーを使用することにした\n環境 Ubuntu Server 24.04 LTS Apt-Cacher-NG/3.7.4 apt-cacher-ngのインストールと設定ファイルの作成 1 sudo apt-get update; sudo apt-get install apt-cacher-ng ; echo \u0026#39;Acquire::http::Proxy \u0026#34;http://127.0.0.1:3142\u0026#34;;\u0026#39; | sudo tee /etc/apt/apt.conf.d/02proxy ; 作成した設定ファイルを編集する\n1 sudo nano /etc/apt/apt.conf.d/02proxy 下記の内容を追記する\n1 Acquire::http::Proxy \u0026#34;http://192.168.10.***:3142\u0026#34;; \u0026ldquo;Acquire::http::Proxy \u0026ldquo;http://192.168.10.16:3142\u0026rdquo;;\u0026rdquo; httpsのものはパススルーをするように設定する\n1 sudo nano /etc/apt-cacher-ng/acng.conf ; 1 PassThroughPattern: .* 設定を読み込むためにサービスを再起動する\n1 sudo service apt-cacher-ng restart WebUIにアクセスする \u0026ldquo;http://192.168.10.16:3142\u0026quot;にアクセスするとWebUIが表示される、どれくらいキャッシュされているか等が見れる\n他のPCからaptのキャッシュを参照するように設定する proxyの設定ファイルを編集する\n1 sudo nano /etc/apt/apt.conf.d/02proxy apt-cacher-ngをインストールしたサーバーのIPアドレスを指定する\n1 Acquire::http::Proxy \u0026#34;http://192.168.10.***:3142\u0026#34;; \u0026ldquo;Acquire::http::Proxy \u0026ldquo;http://192.168.10.16:3142\u0026rdquo;;\u0026rdquo; キャッシュしたデータとログの保存場所 キャッシュの保存場所\n1 /var/cache/apt-cacher-ng/ ログの保存場所\n1 /var/log/apt-cacher-ng/apt-cacher.log キャッシュの削除 ストレージの容量不足でエラーになってしまったため一度キャッシュを削除する\n1 ファイルへの書き込みでエラーが発生しました - write (28: デバイスに空き領域がありません) [IP: 192.168.10.16 3142] キャッシュの削除手順\n容量の確認 1 df -h サービスを一旦止める\n1 sudo systemctl stop apt-cacher-ng キャッシュデータの削除\n1 2 3 sudo rm -rf /var/lib/apt/* \\ sudo rm -rf /var/cache/apt/* \\ sudo rm -rf /var/cache/apt-cacher-ng/ 1 2 sudo mkdir -p /var/cache/apt-cacher-ng/{headers,import,packages,private,temp} sudo chown apt-cacher-ng:apt-cacher-ng -R /var/cache/apt-cacher-ng サービスの再開\n1 2 sudo systemctl start apt-cacher-ng sudo systemctl status apt-cacher-ng 容量に空きができているか確認\n1 df -h 参考URL https://qiita.com/mugimugi/items/edb743c6c32444159384 http://bluearth.cocolog-nifty.com/blog/2020/04/post-182a10.html http://bluearth.cocolog-nifty.com/blog/2020/04/post-8db9b1.html https://qiita.com/mt08/items/c8b8187b1000382d77ae https://zenn.dev/toru3/scraps/07d08e9f86ae71 https://blog.cybozu.io/entry/2016/07/19/103000 直接インストールではなくdockerを使うやり方もあるよう dockerの方が楽そう\napt-cacher-ng で apt をキャッシュする https://zenn.dev/st_little/articles/cache-apt-with-apt-cacher-ng apt-cacher-ng サービスの Docker 化 https://docs.docker.jp/engine/examples/apt-cacher-ng.html ","date":"2024-11-10T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/apt-cacher-ng/","title":"aptのキャッシュをしてくれるapt-cacher-ngのインストール方法"},{"content":"背景 Proxmox上に作成した仮想マシンの容量を10GBで作成したが、足りなくなり、20GBへ容量アップした際の手順です\n環境 Ubuntu Server 24.04 LTS Parted 3.6 ディスク容量：10GB→20GB 参考URL VMware上のUbuntu 22.04 ファイルシステム拡張 https://qiita.com/mcyang/items/a32b914db073f308a3cb 仮想マシンのDiskの拡張手順 下記の順番でコマンドを実行していけば、割り当てられたストレージの容量いっぱいに拡張される パーティションを拡張する 1 sudo apt install lvm2 1 2 3 4 df -h sudo pvdisplay sudo apt install parted sudo parted /dev/sda 1 print free 1 2 3 4 (parted) print free Warning: Not all of the space available to /dev/sda appears to be used, you can fix the GPT to use all of the space (an extra 20971520 blocks) or continue with the current setting? Fix/Ignore? Fix 1 2 3 resizepart 3 100% print free q ※UbuntuDesktopの場合は上記コマンド実行後、下記コマンドを実行するだけで容量が増える 1 sudo resize2fs /dev/sda2 物理ボリュームを拡張する 1 2 3 sudo pvdisplay sudo pvresize /dev/sda3 sudo pvdisplay ロジカルボリュームとファイルシステムを拡張する 1 2 3 sudo lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv sudo resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv df -h 実際の実行結果 10GBから20GBへ増やす 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 mao@mao-cri-o-worker-node:~$ df -h Filesystem Size Used Avail Use% Mounted on tmpfs 795M 1.5M 793M 1% /run /dev/mapper/ubuntu--vg-ubuntu--lv 8.1G 5.7G 2.0G 75% / tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda2 1.7G 94M 1.5G 6% /boot tmpfs 795M 12K 795M 1% /run/user/1000 mao@mao-cri-o-worker-node:~$ sudo pvdisplay [sudo] password for mao: --- Physical volume --- PV Name /dev/sda3 VG Name ubuntu-vg PV Size \u0026lt;8.25 GiB / not usable 0 Allocatable yes (but full) PE Size 4.00 MiB Total PE 2111 Free PE 0 Allocated PE 2111 PV UUID CGNZQy-1SZf-8wIh-yeOq-BIF7-f2VJ-9cWRR2 mao@mao-cri-o-worker-node:~$ sudo apt install parted Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: dmidecode libparted2t64 Suggested packages: libparted-dev libparted-i18n parted-doc The following NEW packages will be installed: dmidecode libparted2t64 parted 0 upgraded, 3 newly installed, 0 to remove and 44 not upgraded. Need to get 267 kB of archives. After this operation, 758 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://jp.archive.ubuntu.com/ubuntu noble/main amd64 dmidecode amd64 3.5-3build1 [72.1 kB] Get:2 http://jp.archive.ubuntu.com/ubuntu noble/main amd64 libparted2t64 amd64 3.6-4build1 [152 kB] Get:3 http://jp.archive.ubuntu.com/ubuntu noble/main amd64 parted amd64 3.6-4build1 [43.3 kB] Fetched 267 kB in 2s (131 kB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package dmidecode. (Reading database ... 72892 files and directories currently installed.) Preparing to unpack .../dmidecode_3.5-3build1_amd64.deb ... Unpacking dmidecode (3.5-3build1) ... Selecting previously unselected package libparted2t64:amd64. Preparing to unpack .../libparted2t64_3.6-4build1_amd64.deb ... Adding \u0026#39;diversion of /lib/x86_64-linux-gnu/libparted.so.2 to /lib/x86_64-linux-gnu/libparted.so.2.usr-is-merged by libparted2t64\u0026#39; Adding \u0026#39;diversion of /lib/x86_64-linux-gnu/libparted.so.2.0.5 to /lib/x86_64-linux-gnu/libparted.so.2.0.5.usr-is-merged by libparted2t64\u0026#39; Unpacking libparted2t64:amd64 (3.6-4build1) ... Selecting previously unselected package parted. Preparing to unpack .../parted_3.6-4build1_amd64.deb ... Unpacking parted (3.6-4build1) ... Setting up dmidecode (3.5-3build1) ... Setting up libparted2t64:amd64 (3.6-4build1) ... Removing \u0026#39;diversion of /lib/x86_64-linux-gnu/libparted.so.2 to /lib/x86_64-linux-gnu/libparted.so.2.usr-is-merged by libparted2t64\u0026#39; Removing \u0026#39;diversion of /lib/x86_64-linux-gnu/libparted.so.2.0.5 to /lib/x86_64-linux-gnu/libparted.so.2.0.5.usr-is-merged by libparted2t64\u0026#39; Setting up parted (3.6-4build1) ... Processing triggers for libc-bin (2.39-0ubuntu8.3) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. mao@mao-cri-o-worker-node:~$ sudo parted /dev/sda GNU Parted 3.6 Using /dev/sda Welcome to GNU Parted! Type \u0026#39;help\u0026#39; to view a list of commands. (parted) print free Warning: Not all of the space available to /dev/sda appears to be used, you can fix the GPT to use all of the space (an extra 20971520 blocks) or continue with the current setting? Fix/Ignore? Fix Model: QEMU QEMU HARDDISK (scsi) Disk /dev/sda: 21.5GB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 17.4kB 1049kB 1031kB Free Space 1 1049kB 2097kB 1049kB bios_grub 2 2097kB 1881MB 1879MB ext4 3 1881MB 10.7GB 8855MB 10.7GB 21.5GB 10.7GB Free Space (parted) resizepart 3 100% (parted) print free Model: QEMU QEMU HARDDISK (scsi) Disk /dev/sda: 21.5GB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 17.4kB 1049kB 1031kB Free Space 1 1049kB 2097kB 1049kB bios_grub 2 2097kB 1881MB 1879MB ext4 3 1881MB 21.5GB 19.6GB (parted) q Information: You may need to update /etc/fstab. mao@mao-cri-o-worker-node:~$ sudo pvdisplay --- Physical volume --- PV Name /dev/sda3 VG Name ubuntu-vg PV Size \u0026lt;8.25 GiB / not usable 0 Allocatable yes (but full) PE Size 4.00 MiB Total PE 2111 Free PE 0 Allocated PE 2111 PV UUID CGNZQy-1SZf-8wIh-yeOq-BIF7-f2VJ-9cWRR2 mao@mao-cri-o-worker-node:~$ sudo pvresize /dev/sda3 Physical volume \u0026#34;/dev/sda3\u0026#34; changed 1 physical volume(s) resized or updated / 0 physical volume(s) not resized mao@mao-cri-o-worker-node:~$ sudo pvdisplay --- Physical volume --- PV Name /dev/sda3 VG Name ubuntu-vg PV Size \u0026lt;18.25 GiB / not usable 16.50 KiB Allocatable yes PE Size 4.00 MiB Total PE 4671 Free PE 2560 Allocated PE 2111 PV UUID CGNZQy-1SZf-8wIh-yeOq-BIF7-f2VJ-9cWRR2 mao@mao-cri-o-worker-node:~$ sudo lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv Size of logical volume ubuntu-vg/ubuntu-lv changed from \u0026lt;8.25 GiB (2111 extents) to \u0026lt;18.25 GiB (4671 extents). Logical volume ubuntu-vg/ubuntu-lv successfully resized. mao@mao-cri-o-worker-node:~$ sudo resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv resize2fs 1.47.0 (5-Feb-2023) Filesystem at /dev/mapper/ubuntu--vg-ubuntu--lv is mounted on /; on-line resizing required old_desc_blocks = 2, new_desc_blocks = 3 The filesystem on /dev/mapper/ubuntu--vg-ubuntu--lv is now 4783104 (4k) blocks long. mao@mao-cri-o-worker-node:~$ df -h Filesystem Size Used Avail Use% Mounted on tmpfs 795M 3.4M 791M 1% /run /dev/mapper/ubuntu--vg-ubuntu--lv 18G 5.7G 12G 34% / tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda2 1.7G 94M 1.5G 6% /boot tmpfs 795M 12K 795M 1% /run/user/1000 mao@mao-cri-o-worker-node:~$ ","date":"2024-11-09T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/ubuntu-server-disk-expansion/","title":"仮想マシン上のDiskの拡張手順"},{"content":"環境 harbor v2.11.1 docker version 27.3.1 Docker Compose version v2.29.7 OpenSSL 3.0.13 インストール要件確認\nhttps://goharbor.io/docs/2.11.0/install-config/installation-prereqs/ harborのオフラインインストーラーとdocker,docker compose,OpenSSLが必要なのでダウンロードとインストールをしていきます\nofflineインストーラーをダウンロードする 最新版を確認しダウンロードする\nhttps://github.com/goharbor/harbor/releases 2024年10月13日時点での最新版の2.11.1をダウンロードします\n1 wget https://github.com/goharbor/harbor/releases/download/v2.11.1/harbor-offline-installer-v2.11.1.tgz ダウンロードしたファイルを解凍して設定ファイルをコピーする\n1 2 3 tar xvzf harbor-offline-installer-v2.11.1.tgz cd harbor cp harbor.yml.tmpl harbor.yml 1 2 3 4 5 6 7 8 9 10 mao@harbor-server:~$ tar xvzf harbor-offline-installer-v2.11.1.tgz harbor/harbor.v2.11.1.tar.gz harbor/prepare harbor/LICENSE harbor/install.sh harbor/common.sh harbor/harbor.yml.tmpl mao@harbor-server:~$ cd harbor mao@harbor-server:~/harbor$ cp harbor.yml.tmpl harbor.yml mao@harbor-server:~/harbor$ 設定ファイルを編集する\n\u0026ldquo;hostname:\u0026ldquo;をIPアドレスへ変更する \u0026ldquo;https:\u0026ldquo;の設定を、今回はhttpでアクセスするのでコメントアウトする \u0026ldquo;certificate: /your/certificate/path\u0026quot;もコメントアウトする \u0026ldquo;private_key: /your/private/key/path\u0026quot;もコメントアウトする 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # Configuration file of Harbor # The IP address or hostname to access admin UI and registry service. # DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients. + hostname: 192.168.10.24 - hostname: reg.mydomain.com # http related config http: # port for http, default is 80. If https enabled, this port will redirect to https port port: 80 # https related config + #https: - https: # https port for harbor, default is 443 + #port: 443 - port: 443 # The path of cert and key files for nginx + #certificate: /your/certificate/path + #private_key: /your/private/key/path - certificate: /your/certificate/path - private_key: /your/private/key/path # enable strong ssl ciphers (default: false) # strong_ssl_ciphers: false docker engineとdocker composeをインストールする 公式ドキュメントの通りにインストールする\nhttps://docs.docker.com/engine/install/ubuntu/#install-using-the-repository 1 2 3 4 5 6 7 8 9 10 11 12 13 # Add Docker\u0026#39;s official GPG key: sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources: echo \\ \u0026#34;deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \\ $(. /etc/os-release \u0026amp;\u0026amp; echo \u0026#34;$VERSION_CODENAME\u0026#34;) stable\u0026#34; | \\ sudo tee /etc/apt/sources.list.d/docker.list \u0026gt; /dev/null sudo apt-get update 1 sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin OpenSSLがインストールされているか確認する 1 openssl version 1 2 3 mao@harbor-server:~$ openssl version OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024) mao@harbor-server:~$ harborをインストールする 下記のコマンドを実行してインストールする\n1 sudo ./install.sh 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 mao@harbor-server:~/harbor$ sudo ./install.sh [Step 0]: checking if docker is installed ... Note: docker version: 27.3.1 [Step 1]: checking docker-compose is installed ... Note: Docker Compose version v2.29.7 [Step 2]: loading Harbor images ... fa65d0b345aa: Loading layer 40.5MB/40.5MB 40dbf36fc7e0: Loading layer 112.7MB/112.7MB e27e903fe711: Loading layer 51.64MB/51.64MB 312f4e210c8d: Loading layer 13.78MB/13.78MB 3e3d5fae203b: Loading layer 65.54kB/65.54kB bd56ad4f7c00: Loading layer 2.56kB/2.56kB 58bd888248a1: Loading layer 1.536kB/1.536kB 8ad07d6448d6: Loading layer 12.29kB/12.29kB 7604bd93290d: Loading layer 2.744MB/2.744MB 8637aeee1380: Loading layer 492.5kB/492.5kB Loaded image: goharbor/prepare:v2.11.1 59cd002b46d2: Loading layer 21.86MB/21.86MB 2e8f9fa1e5f5: Loading layer 175MB/175MB ecd34246c904: Loading layer 26.04MB/26.04MB d8b960cafd25: Loading layer 18.54MB/18.54MB 410dc4347a57: Loading layer 5.12kB/5.12kB 80921caabb24: Loading layer 6.144kB/6.144kB e91542fda4dd: Loading layer 3.072kB/3.072kB df3f2e9dd439: Loading layer 2.048kB/2.048kB d8facbd2a6c0: Loading layer 2.56kB/2.56kB 4715dde7127c: Loading layer 7.68kB/7.68kB Loaded image: goharbor/harbor-db:v2.11.1 926647c50af4: Loading layer 17.23MB/17.23MB 99ff9f9dc8ce: Loading layer 28.75MB/28.75MB 99078c9b3a60: Loading layer 4.608kB/4.608kB fe5588cde585: Loading layer 29.54MB/29.54MB Loaded image: goharbor/harbor-exporter:v2.11.1 4ec814cdc7b2: Loading layer 21.86MB/21.86MB 235f2878bf8a: Loading layer 110.5MB/110.5MB cdccfb99123c: Loading layer 3.072kB/3.072kB c7ea796bb849: Loading layer 59.9kB/59.9kB f8a27040ef0d: Loading layer 61.95kB/61.95kB Loaded image: goharbor/redis-photon:v2.11.1 7a130cf406bb: Loading layer 121.1MB/121.1MB Loaded image: goharbor/nginx-photon:v2.11.1 7786af5594f6: Loading layer 121.1MB/121.1MB 0c39daf00027: Loading layer 6.703MB/6.703MB c9af590a487f: Loading layer 251.9kB/251.9kB 9ba79732c750: Loading layer 1.477MB/1.477MB Loaded image: goharbor/harbor-portal:v2.11.1 2124fec7bf7d: Loading layer 17.23MB/17.23MB 257165566506: Loading layer 3.584kB/3.584kB 71c6cf01ef4c: Loading layer 2.56kB/2.56kB e6aaf52bc017: Loading layer 67.13MB/67.13MB ac2b2a90f17c: Loading layer 5.632kB/5.632kB 2deff795bee3: Loading layer 125.4kB/125.4kB e4bd545de86d: Loading layer 201.7kB/201.7kB 847012124c72: Loading layer 68.25MB/68.25MB d1601b055891: Loading layer 2.56kB/2.56kB Loaded image: goharbor/harbor-core:v2.11.1 e4f7bca07127: Loading layer 130.8MB/130.8MB 3d744fdec5a0: Loading layer 3.584kB/3.584kB e2c98f9cef30: Loading layer 3.072kB/3.072kB cbe22372d70a: Loading layer 2.56kB/2.56kB c3cc060f064c: Loading layer 3.072kB/3.072kB 184ad5ccf4f4: Loading layer 3.584kB/3.584kB 4a30d6215ed7: Loading layer 20.48kB/20.48kB Loaded image: goharbor/harbor-log:v2.11.1 d2e836032dca: Loading layer 17.23MB/17.23MB 6159b9476a38: Loading layer 3.584kB/3.584kB 6cd40121c7f9: Loading layer 2.56kB/2.56kB ab578d976e3e: Loading layer 54.27MB/54.27MB 74d4b342c232: Loading layer 55.06MB/55.06MB Loaded image: goharbor/harbor-jobservice:v2.11.1 a370043a2cd6: Loading layer 14.22MB/14.22MB 068c345c0269: Loading layer 4.096kB/4.096kB 24607b1b1b88: Loading layer 17.86MB/17.86MB d460b7320fa0: Loading layer 3.072kB/3.072kB 41f6293d43da: Loading layer 38.93MB/38.93MB 47c258cefc9f: Loading layer 57.57MB/57.57MB Loaded image: goharbor/harbor-registryctl:v2.11.1 b020161dfc96: Loading layer 14.22MB/14.22MB 660cc2bb7fc2: Loading layer 4.096kB/4.096kB 093817c1779d: Loading layer 3.072kB/3.072kB baa5b276e894: Loading layer 17.86MB/17.86MB 4db5e5303fdc: Loading layer 18.65MB/18.65MB Loaded image: goharbor/registry-photon:v2.11.1 cf045d0bacdb: Loading layer 14.73MB/14.73MB 7b3be75d25ec: Loading layer 4.096kB/4.096kB 300144cef16c: Loading layer 3.072kB/3.072kB 20b0983274b3: Loading layer 127.1MB/127.1MB c64d3b51f3b9: Loading layer 14.89MB/14.89MB ecf40289f004: Loading layer 142.7MB/142.7MB Loaded image: goharbor/trivy-adapter-photon:v2.11.1 [Step 3]: preparing environment ... [Step 4]: preparing harbor configs ... prepare base dir is set to /home/mao/harbor ERROR:root:Error: The protocol is https but attribute ssl_cert is not set Error happened in config validation... mao@harbor-server:~/harbor$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 mao@harbor-server:~/harbor$ sudo ./install.sh [Step 0]: checking if docker is installed ... Note: docker version: 27.3.1 [Step 1]: checking docker-compose is installed ... Note: Docker Compose version v2.29.7 [Step 2]: loading Harbor images ... Loaded image: goharbor/prepare:v2.11.1 Loaded image: goharbor/harbor-db:v2.11.1 Loaded image: goharbor/harbor-exporter:v2.11.1 Loaded image: goharbor/redis-photon:v2.11.1 Loaded image: goharbor/nginx-photon:v2.11.1 Loaded image: goharbor/harbor-portal:v2.11.1 Loaded image: goharbor/harbor-core:v2.11.1 Loaded image: goharbor/harbor-log:v2.11.1 Loaded image: goharbor/harbor-jobservice:v2.11.1 Loaded image: goharbor/harbor-registryctl:v2.11.1 Loaded image: goharbor/registry-photon:v2.11.1 Loaded image: goharbor/trivy-adapter-photon:v2.11.1 [Step 3]: preparing environment ... [Step 4]: preparing harbor configs ... prepare base dir is set to /home/mao/harbor WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https Generated configuration file: /config/portal/nginx.conf Generated configuration file: /config/log/logrotate.conf Generated configuration file: /config/log/rsyslog_docker.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/registryctl/config.yml Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml Generated and saved secret to file: /data/secret/keys/secretkey Successfully called func: create_root_cert Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir Note: stopping existing Harbor instance ... WARN[0000] /home/mao/harbor/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion [Step 5]: starting Harbor ... WARN[0000] /home/mao/harbor/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion [+] Running 10/10 ✔ Network harbor_harbor Created 0.1s ✔ Container harbor-log Started 0.7s ✔ Container registry Started 0.9s ✔ Container harbor-portal Started 1.0s ✔ Container harbor-db Started 1.1s ✔ Container registryctl Started 1.1s ✔ Container redis Started 1.0s ✔ Container harbor-core Started 1.1s ✔ Container harbor-jobservice Started 1.4s ✔ Container nginx Started 1.4s ✔ ----Harbor has been installed and started successfully.---- mao@harbor-server:~/harbor$ インストール後に自動で下記のようなフォルダ構成になる\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 mao@harbor-server:~/harbor$ tree . ├── LICENSE ├── common │ └── config │ ├── core │ │ ├── app.conf │ │ ├── certificates │ │ └── env │ ├── db │ │ └── env │ ├── jobservice │ │ ├── config.yml │ │ └── env │ ├── log │ │ ├── logrotate.conf │ │ └── rsyslog_docker.conf │ ├── nginx │ │ ├── conf.d │ │ └── nginx.conf │ ├── portal │ │ └── nginx.conf │ ├── registry │ │ ├── config.yml │ │ ├── passwd │ │ └── root.crt │ ├── registryctl │ │ ├── config.yml │ │ └── env │ └── shared │ └── trust-certificates ├── common.sh ├── docker-compose.yml ├── harbor.v2.11.1.tar.gz ├── harbor.yml ├── harbor.yml.tmpl ├── install.sh └── prepare 15 directories, 22 files mao@harbor-server:~/harbor$ WebUIにIPアドレスでアクセスし下記のアカウント情報でログインする\nUsername admin Password Harbor12345 harborのWebUI上でプロジェクトを作成する WebUIでログインして、左のメニューバーから\u0026quot;Project\u0026quot;を押す 最初は\u0026quot;library\u0026quot;があるが、今回はプロジェクトを新規作成する\n\u0026ldquo;NEW PROJECT\u0026quot;を押す \u0026ldquo;Project Name\u0026quot;を今回は\u0026quot;homelab\u0026quot;にする \u0026ldquo;Public\u0026quot;にチェックを入れておく \u0026ldquo;OK\u0026quot;を押すとプロジェクトを作成される harborにdocker imageをpushする ログインする\n1 docker login 192.168.10.24 ログインしようとするとhttpsではないのでエラーになる\n1 2 3 4 5 6 mao@harbor-server:~/harbor$ docker login 192.168.10.24 Username: admin Password: time=\u0026#34;2024-10-13T01:22:33Z\u0026#34; level=info msg=\u0026#34;Error logging in to endpoint, trying next endpoint\u0026#34; error=\u0026#34;Get \\\u0026#34;https://192.168.10.24/v2/\\\u0026#34;: dial tcp 192.168.10.24:443: connect: connection refused\u0026#34; Get \u0026#34;https://192.168.10.24/v2/\u0026#34;: dial tcp 192.168.10.24:443: connect: connection refused mao@harbor-server:~/harbor$ httpで接続できるように設定する ファイルを作成する\ndaemon.json 1 2 3 4 5 { \u0026#34;insecure-registries\u0026#34;: [ \u0026#34;192.168.10.24\u0026#34; ] } 設定ファイルをコピーしてdockerを再起動する\n1 2 3 sudo cp -f ./daemon.json /etc/docker/daemon.json sudo systemctl restart docker sudo systemctl daemon-reload httpでdockerにログインする リロードしたあとにそのままだとログインできなかったので、一度docker composeを再起動する\n1 2 sudo docker compose down --volumes sudo docker compose up -d ログインする\n1 sudo docker login 192.168.10.24 Username admin Password Harbor12345 1 2 3 4 5 6 7 8 9 mao@harbor-server:~/harbor$ sudo docker login 192.168.10.24 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credential-stores Login Succeeded mao@harbor-server:~/harbor$ docker imageをDockerHubからpullしてharborにpushする DockerHubからimageをpullする\n例として\u0026quot;traefik:3.1.6\u0026quot;と\u0026quot;debian:12.7\u0026quot;をpullしてpushする 1 2 sudo docker pull traefik:3.1.6 sudo docker pull debian:12.7 pullしてimageにtagを付ける\n1 2 sudo docker tag traefik:3.1.6 192.168.10.24/homelab/homelab-traefik:3.1.6 sudo docker tag debian:12.7 192.168.10.24/homelab/homelab-debian:12.7 harborにimageをpushする\n1 2 sudo docker push 192.168.10.24/homelab/homelab-traefik:3.1.6 sudo docker push 192.168.10.24/homelab/homelab-debian:12.7 imageを削除する\n1 2 3 4 5 6 7 sudo docker image ls sudo docker rmi traefik:3.1.6 sudo docker rmi 192.168.10.24/homelab/homelab-traefik:3.1.6 sudo docker rmi debian:12.7 sudo docker rmi 192.168.10.24/homelab/homelab-debian:12.7 sudo docker image ls 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 mao@harbor-server:~/harbor$ sudo docker pull traefik:3.1.6 3.1.6: Pulling from library/traefik 43c4264eed91: Pull complete f60fb4c0fbec: Pull complete 9a6d31097c4f: Pull complete e5f06ee63d76: Pull complete Digest: sha256:f703a2ac2ddf75f6e06c9cccac7f158765e2ab42d642b04e79e2e3d7355c2ddc Status: Downloaded newer image for traefik:3.1.6 docker.io/library/traefik:3.1.6 mao@harbor-server:~/harbor$ sudo docker tag traefik:3.1.6 192.168.10.24/homelab/homelab-traefik:3.1.6 mao@harbor-server:~/harbor$ sudo docker push 192.168.10.24/homelab/homelab-traefik:3.1.6 The push refers to repository [192.168.10.24/homelab/homelab-traefik] affde95bf5bb: Pushed 9a1c58574d55: Pushed 96513adaf6d9: Pushed 63ca1fbb43ae: Pushed 3.1.6: digest: sha256:22aec04848987fe5b3999a4099d766de614b04da52a936fc5ac214ffec04dbac size: 1157 mao@harbor-server:~/harbor$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 mao@harbor-server:~/harbor$ sudo docker pull debian:12.7 12.7: Pulling from library/debian cdd62bf39133: Pull complete Digest: sha256:27586f4609433f2f49a9157405b473c62c3cb28a581c413393975b4e8496d0ab Status: Downloaded newer image for debian:12.7 docker.io/library/debian:12.7 mao@harbor-server:~/harbor$ sudo docker tag debian:12.7 192.168.10.24/homelab/homelab-debian:12.7 mao@harbor-server:~/harbor$ sudo docker push 192.168.10.24/homelab/homelab-debian:12.7 The push refers to repository [192.168.10.24/homelab/homelab-debian] d50132f2fe78: Pushed 12.7: digest: sha256:e225d70fafe80791f18c79b8d76afa1d1b4192b3a40a50f1ffd4de84555ebd04 size: 529 mao@harbor-server:~/harbor$ sudo docker rmi debian:12.7 Untagged: debian:12.7 Untagged: debian@sha256:27586f4609433f2f49a9157405b473c62c3cb28a581c413393975b4e8496d0ab mao@harbor-server:~/harbor$ sudo docker rmi 192.168.10.24/homelab/homelab-debian:12.7 Untagged: 192.168.10.24/homelab/homelab-debian:12.7 Untagged: 192.168.10.24/homelab/homelab-debian@sha256:e225d70fafe80791f18c79b8d76afa1d1b4192b3a40a50f1ffd4de84555ebd04 Deleted: sha256:c7f9867d67219411cb7dff983c83eddb881d415eae5495ad9116fb5d6db8bb60 Deleted: sha256:d50132f2fe78aaa8e67f229e53305d0c4a7a86c9deda4bf3160be0a678926458 mao@harbor-server:~/harbor$ sudo docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE goharbor/harbor-exporter v2.11.1 cdf68efc001e 8 weeks ago 114MB goharbor/redis-photon v2.11.1 acf90a312d47 8 weeks ago 170MB goharbor/trivy-adapter-photon v2.11.1 24a8273e807a 8 weeks ago 339MB goharbor/harbor-registryctl v2.11.1 43fca2a06374 8 weeks ago 168MB goharbor/registry-photon v2.11.1 9da6663b36f2 8 weeks ago 90.3MB goharbor/nginx-photon v2.11.1 193a1b77b7d4 8 weeks ago 159MB goharbor/harbor-log v2.11.1 2752e033bfbb 8 weeks ago 169MB goharbor/harbor-jobservice v2.11.1 a8005a88b3dc 8 weeks ago 165MB goharbor/harbor-core v2.11.1 eaf65baad3f6 8 weeks ago 191MB goharbor/harbor-portal v2.11.1 f58813018a49 8 weeks ago 167MB goharbor/harbor-db v2.11.1 be56f8030c48 8 weeks ago 277MB goharbor/prepare v2.11.1 1d00ffdb2e67 8 weeks ago 216MB mao@harbor-server:~/harbor$ WebUI上でもイメージがpushされているか確認する harborからdocker imageをpullする 下記のコマンドでharborからpullしてみる\n1 2 sudo docker pull 192.168.10.24/homelab/homelab-traefik:3.1.6 sudo docker pull 192.168.10.24/homelab/homelab-debian:12.7 pullしようとしているdocker imageがないことを確認する\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 mao@harbor-server:~$ sudo docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE goharbor/harbor-exporter v2.11.1 cdf68efc001e 8 weeks ago 114MB goharbor/redis-photon v2.11.1 acf90a312d47 8 weeks ago 170MB goharbor/trivy-adapter-photon v2.11.1 24a8273e807a 8 weeks ago 339MB goharbor/harbor-registryctl v2.11.1 43fca2a06374 8 weeks ago 168MB goharbor/registry-photon v2.11.1 9da6663b36f2 8 weeks ago 90.3MB goharbor/nginx-photon v2.11.1 193a1b77b7d4 8 weeks ago 159MB goharbor/harbor-log v2.11.1 2752e033bfbb 8 weeks ago 169MB goharbor/harbor-jobservice v2.11.1 a8005a88b3dc 8 weeks ago 165MB goharbor/harbor-core v2.11.1 eaf65baad3f6 8 weeks ago 191MB goharbor/harbor-portal v2.11.1 f58813018a49 8 weeks ago 167MB goharbor/harbor-db v2.11.1 be56f8030c48 8 weeks ago 277MB goharbor/prepare v2.11.1 1d00ffdb2e67 8 weeks ago 216MB ないことを確認したのでdocker imageをpullする\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 mao@harbor-server:~$ sudo docker pull 192.168.10.24/homelab/homelab-traefik:3.1.6 3.1.6: Pulling from homelab/homelab-traefik 43c4264eed91: Pull complete f60fb4c0fbec: Pull complete 9a6d31097c4f: Pull complete e5f06ee63d76: Pull complete Digest: sha256:22aec04848987fe5b3999a4099d766de614b04da52a936fc5ac214ffec04dbac Status: Downloaded newer image for 192.168.10.24/homelab/homelab-traefik:3.1.6 192.168.10.24/homelab/homelab-traefik:3.1.6 mao@harbor-server:~$ sudo docker pull 192.168.10.24/homelab/homelab-debian:12.7 12.7: Pulling from homelab/homelab-debian cdd62bf39133: Pull complete Digest: sha256:e225d70fafe80791f18c79b8d76afa1d1b4192b3a40a50f1ffd4de84555ebd04 Status: Downloaded newer image for 192.168.10.24/homelab/homelab-debian:12.7 192.168.10.24/homelab/homelab-debian:12.7 ちゃんとpullできているか確認する\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 mao@harbor-server:~$ sudo docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.10.24/homelab/homelab-traefik 3.1.6 ec31dae6a8fb 4 days ago 174MB 192.168.10.24/homelab/homelab-debian 12.7 c7f9867d6721 2 weeks ago 117MB goharbor/harbor-exporter v2.11.1 cdf68efc001e 8 weeks ago 114MB goharbor/redis-photon v2.11.1 acf90a312d47 8 weeks ago 170MB goharbor/trivy-adapter-photon v2.11.1 24a8273e807a 8 weeks ago 339MB goharbor/harbor-registryctl v2.11.1 43fca2a06374 8 weeks ago 168MB goharbor/registry-photon v2.11.1 9da6663b36f2 8 weeks ago 90.3MB goharbor/nginx-photon v2.11.1 193a1b77b7d4 8 weeks ago 159MB goharbor/harbor-log v2.11.1 2752e033bfbb 8 weeks ago 169MB goharbor/harbor-jobservice v2.11.1 a8005a88b3dc 8 weeks ago 165MB goharbor/harbor-core v2.11.1 eaf65baad3f6 8 weeks ago 191MB goharbor/harbor-portal v2.11.1 f58813018a49 8 weeks ago 167MB goharbor/harbor-db v2.11.1 be56f8030c48 8 weeks ago 277MB goharbor/prepare v2.11.1 1d00ffdb2e67 8 weeks ago 216MB mao@harbor-server:~$ しっかりと\u0026quot;homelab-traefik\u0026quot;と\u0026quot;homelab-debian\u0026quot;が増えているのでpullできている\n以上でharborを使用して、自分専用のローカルリポジトリが作成できた\n今回は仮想マシン上に構築したが、Kubernetes上にも構築できるようなので、そちらもやってみようと思う\n参考URL OSSのコンテナレジストリ「Harbor」インストール手順 https://tech-mmmm.blogspot.com/2022/12/ossharbor.html docker-composeでHarborをインストールする https://zenn.dev/vampire_yuta/articles/5fc115fcfd083f プライベートコンテナリポジトリを構築しdockerコンテナを起動してみる https://qiita.com/watav/items/bac6564f9fb38578a67b Harbor(プライベートコンテナレジストリ)のインストール https://engineeroutput.com/harbor%E3%83%97%E3%83%A9%E3%82%A4%E3%83%99%E3%83%BC%E3%83%88%E3%82%B3%E3%83%B3%E3%83%86%E3%83%8A%E3%83%AC%E3%82%B8%E3%82%B9%E3%83%88%E3%83%AA%E3%81%AE%E3%82%A4%E3%83%B3%E3%82%B9%E3%83%88%E3%83%BC/ ","date":"2024-10-15T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/harbor-install/","title":"Harborのインストールしてpushとpullをしてみる"},{"content":"Ubuntuにスワップを作成する（ラズパイ） ラズパイのスペックは下記の通りです\nRaspberry Pi 3B メモリ1GB SDカード32GB メモリが少ないのでDockerを使用するには少ないので、8GBのスワップメモリを作成します\n何も表示されない場合はスワップは作成されていない 1 sudo swapon --show ストレージの容量を確認します 1 df -h スワップ用のファイルを作成する 8GBのスワップを作成する 1 2 sudo fallocate -l 8G /swapfile ls -lh /swapfile スワップファイルを設定して、有効にします 1 2 3 sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile 再度確認をします 1 sudo swapon --show スワップが作成されています 1 2 3 mao@raspberrypi3:~$ sudo swapon --show NAME TYPE SIZE USED PRIO /swapfile file 8G 0B -2 スワップの永続設定をする 1 sudo nano /etc/fstab 下記を追記する 1 /swapfile none swap sw 0 0 UbuntuにDockerをインストールする 公式ドキュメントを参考にインストールする\nhttps://docs.docker.com/engine/install/ubuntu/ debをダウンロードする 2024年10月05日時点での最新版をダウンロードする amd64ではなくarm64をダウンロードする 1 2 3 4 5 wget https://download.docker.com/linux/ubuntu/dists/noble/pool/stable/arm64/containerd.io_1.7.22-1_arm64.deb \\ wget https://download.docker.com/linux/ubuntu/dists/noble/pool/stable/arm64/docker-buildx-plugin_0.17.1-1~ubuntu.24.04~noble_arm64.deb \\ wget https://download.docker.com/linux/ubuntu/dists/noble/pool/stable/arm64/docker-ce-cli_27.3.1-1~ubuntu.24.04~noble_arm64.deb \\ wget https://download.docker.com/linux/ubuntu/dists/noble/pool/stable/arm64/docker-ce_27.3.1-1~ubuntu.24.04~noble_arm64.deb \\ wget https://download.docker.com/linux/ubuntu/dists/noble/pool/stable/arm64/docker-compose-plugin_2.29.7-1~ubuntu.24.04~noble_arm64.deb インストールをする 1 2 3 4 5 sudo dpkg -i ./containerd.io_\u0026lt;version\u0026gt;_\u0026lt;arch\u0026gt;.deb \\ ./docker-ce_\u0026lt;version\u0026gt;_\u0026lt;arch\u0026gt;.deb \\ ./docker-ce-cli_\u0026lt;version\u0026gt;_\u0026lt;arch\u0026gt;.deb \\ ./docker-buildx-plugin_\u0026lt;version\u0026gt;_\u0026lt;arch\u0026gt;.deb \\ ./docker-compose-plugin_\u0026lt;version\u0026gt;_\u0026lt;arch\u0026gt;.deb 1 2 3 4 5 sudo dpkg -i ./containerd.io_1.7.22-1_arm64.deb \\ ./docker-ce_27.3.1-1~ubuntu.24.04~noble_arm64.deb \\ ./docker-ce-cli_27.3.1-1~ubuntu.24.04~noble_arm64.deb \\ ./docker-buildx-plugin_0.17.1-1~ubuntu.24.04~noble_arm64.deb \\ ./docker-compose-plugin_2.29.7-1~ubuntu.24.04~noble_arm64.deb containerd.io_1.7.22-1_arm64.deb docker-ce_27.3.1-1-ubuntu.24.04~noble_arm64.deb docker-ce-cli_27.3.1-1-ubuntu.24.04~noble_arm64.deb docker-buildx-plugin_0.17.1-1-ubuntu.24.04~noble_arm64.deb docker-compose-plugin_2.29.7-1-ubuntu.24.04~noble_arm64.deb バージョン確認 dockerとdocker composeのバージョンを確認する\n1 2 sudo docker version sudo docker compose version Cloudflare Tunnelを作成する トンネルをCloudflareのサイトで作成する Cloudflareのホーム画面の左側のメニューから\u0026quot;Zero Trust\u0026quot;を押します その後、左側のメニューから\u0026quot;Networks\u0026quot;\u0026gt;\u0026ldquo;Tunnels\u0026quot;を押します \u0026ldquo;Tunnels\u0026quot;の画面になったら\u0026quot;\u0026ldquo;を押してTunnelを作成します \u0026ldquo;Create a tunnel\u0026quot;と表示された画面になります \u0026ldquo;Select your tunnel type\u0026quot;は\u0026quot;Cloudflared\u0026quot;を選択して\u0026quot;Next\u0026quot;を押します \u0026ldquo;Name your tunnel\u0026quot;に自分がわかるようにTunnelの名前を入力し、\u0026ldquo;Save Tunnel\u0026quot;を押します すると\u0026quot;Configure\u0026quot;ｔ表示された画面になります \u0026ldquo;Choose your environment\u0026quot;は\u0026quot;Docker\u0026quot;を選択します \u0026ldquo;Install and run a connector\u0026quot;にdockerコマンドとトークンが表示されるので、コピーしておきます トークンは他の人には絶対に見せないでください（トークンがあれば接続できてしまいます） コピーをしたら\u0026quot;Next\u0026quot;を押します \u0026ldquo;Add public hostname for TunnelName\u0026quot;と表示された画面になります \u0026ldquo;Subdomain\u0026quot;にはサブドメイン名を入力します \u0026ldquo;Domain\u0026quot;にはCloudflareで管理されているドメインが選択できます \u0026ldquo;Service\u0026quot;の\u0026quot;Type\u0026quot;にはHTTPやSSHなどを選択できます \u0026ldquo;URL\u0026quot;にはdocker composeの場合はコンテナ名とポートを入力し、\u0026ldquo;Save tunnel\u0026quot;を押します （例）Type:\u0026ldquo;HTTP\u0026rdquo;,Service:\u0026ldquo;nginx:8080\u0026rdquo; \u0026ldquo;Tunnels\u0026quot;と表示された画面に戻ります、以上でトンネルが作成されます dokcer composeファイルの作成と起動・停止 compose.yamlファイルの作成をする\ncompose.yaml 1 2 3 4 5 6 7 8 9 10 services: blog: image: nginx:latest tunnel: restart: always image: cloudflare/cloudflared:latest command: tunnel --no-autoupdate run environment: - TUNNEL_TOKEN=トンネル作成時のトークンをここに記載する 起動と停止のコマンド\n1 2 sudo docker compose up -d sudo docker compose down --volumes アクセスする docker composeで起動し、設定したサブドメインにアクセスして画面が表示されれば無事起動されトンネルができている\n下記の画像は\u0026quot;raspberrypi3b\u0026quot;をサブドメインとして設定した\n503等がでると、cloudflaredは起動しているがdocker compose内での接続が上手く行っていない可能性がある\n※コンテナ名やポート番号が異なるなど 参考URL MisskeyをDocker Compose+Cloudflare Tunnelでサクッと建てる https://zenn.dev/hrko/scraps/29df6c7ac02f03 個人的docker composeおすすめtips 9選 https://future-architect.github.io/articles/20240620a/ cloudflare tunnelで自宅サーバー（動的IP）を運用したい、しかもdocker composeで https://qiita.com/github0013@github/items/6686cc39a297b521d58e ","date":"2024-10-07T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/cloudflare-tunnel-raspberrypi3b/","title":"CloudflareTunnelとDockerComposeを使用してラズパイ上に構築したサイトにアクセスできるようにする"},{"content":"試験について 基本情報技術者試験 試験日_2024年09月30日 科目A_690点 科目B_625点 なぜ取得しようと思ったのか 基本的な知識をしっかり勉強して視野を広げたかったのと、知識の証明として資格を取得したかったからです。\n使用した教材・サイト 科目A 基本情報技術者試験ドットコム、過去問道場 https://www.fe-siken.com/fekakomon.php 【2024年】基本情報技術者試験の科目Aを12時間で完璧に理解 https://www.youtube.com/watch?v=oqaBEnhIxk0 科目B 情報処理教科書 出るとこだけ！基本情報技術者［科目B］第4版 https://www.amazon.co.jp/dp/4798182524 基本情報技術者試験ドットコム、過去問道場 https://www.fe-siken.com/fekakomon.php Youtube 【科目B】アルゴリズム問題をたった1動画で対策_基本情報技術者試験 https://www.youtube.com/watch?v=wFpyeWto8Og IPA(情報処理推進機構)、サンプル問題 https://www.ipa.go.jp/shiken/syllabus/henkou/2022/20220425.html 勉強してみて感じたこと 科目Aの勉強をしていて聞いたことある単語だが、意味を知らなかったり、他の単語と混同したりしていたのを勉強していくことで正しい意味を理解していけました。\n少しずつ単語と意味を頭の中で紐付いて来たことで、問題が解けるようになっていきました。 解けるようになっていくと自分自身がレベルアップしているような感覚になり、どんどん楽しくなっていきました。\n試験勉強をしたことで自分の中では、当初の目標通りITに関する視野が広げることができました。\n結果として試験も点数ギリギリですが合格することができ、資格を取得するという目標も達成することができました。\n次の目標 LPIC1と応用情報技術者試験を次の目標にしています。\nLPIC1はKubernetesを触っていて、Linuxをより知りたいと思ったからです。\nそのための第一歩としてLPIC1を取得しようと思っています。\n応用情報技術者試験は、基本情報技術者試験の次のステップとしてと、もっと深くITについて勉強したいからです。\n","date":"2024-10-05T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/fe-passed-20240930/","title":"基本情報技術者試験に合格しました"},{"content":"環境 Kubernetes 1.31.0 Control-Plane：1台 Woker-Node：3台+1台 cri-o v1.30.5 Helm v3.15.4 Cilium v1.16.1 Kubernetesの準備 cri-oインストール 1 2 curl -fsSL https://pkgs.k8s.io/addons:/cri-o:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/cri-o-apt-keyring.gpg 1 2 echo \u0026#34;deb [signed-by=/etc/apt/keyrings/cri-o-apt-keyring.gpg] https://pkgs.k8s.io/addons:/cri-o:/stable:/v1.30/deb/ /\u0026#34; | sudo tee /etc/apt/sources.list.d/cri-o.list 1 2 3 4 sudo apt update \u0026amp;\u0026amp; sudo apt install cri-o \u0026amp;\u0026amp; sudo systemctl daemon-reload \u0026amp;\u0026amp; systemctl enable --now crio 1 crio version 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 mao@cilium-woker-node-01:~$ crio version INFO[2024-09-04 11:27:18.691313588Z] Starting CRI-O, version: 1.30.5, git: df27b8f8eb49a13c522aca56ee4ec27bc7482fad(clean) Version: 1.30.5 GitCommit: df27b8f8eb49a13c522aca56ee4ec27bc7482fad GitCommitDate: 2024-09-02T07:15:35Z GitTreeState: clean BuildDate: 1970-01-01T00:00:00Z GoVersion: go1.22.0 Compiler: gc Platform: linux/amd64 Linkmode: static BuildTags: static netgo osusergo exclude_graphdriver_btrfs exclude_graphdriver_devicemapper seccomp apparmor selinux LDFlags: unknown SeccompEnabled: true AppArmorEnabled: false mao@cilium-woker-node-01:~$ スワップをOFFにする 1 2 3 sudo swapoff -a sudo nano /etc/fstab free -h カーネルパラメータの設定をする 1 2 3 4 cat \u0026lt;\u0026lt;EOF | sudo tee /etc/modules-load.d/k8s.conf overlay br_netfilter EOF 1 2 sudo modprobe overlay \u0026amp;\u0026amp; sudo modprobe br_netfilter 1 2 3 4 5 cat \u0026lt;\u0026lt;EOF | sudo tee /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 EOF 1 sudo sysctl --system crioの設定をする 1 2 sudo crio config default | sudo tee /etc/crio/crio.conf sudo nano /etc/crio/crio.conf 1 2 3 4 5 [crio.runtime] conmon_cgroup = \u0026#34;pod\u0026#34; cgroup_manager = \u0026#34;cgroupfs\u0026#34; default_runtime = \u0026#34;runc\u0026#34; 1 2 [crio.image] pause_image = \u0026#34;registry.k8s.io/pause:3.9\u0026#34; 1 sudo systemctl restart cri-o runCのインストール 1 2 sudo wget https://github.com/opencontainers/runc/releases/download/v1.1.14/runc.amd64 sudo install -m 755 runc.amd64 /usr/local/sbin/runc kubelet,kubeadm,kubectlのインストール 1 2 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list 1 2 3 sudo apt update \u0026amp;\u0026amp; sudo apt install kubelet kubeadm kubectl \u0026amp;\u0026amp; sudo apt-mark hold kubelet kubeadm kubectl Control-Planeでの作業 1 sudo kubeadm init --apiserver-advertise-address=192.168.10.55 --pod-network-cidr=10.128.0.0/16 1 2 3 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config 1 sudo kubeadm token create --print-join-command Woker-Nodeでの作業 1 2 sudo kubeadm join 192.168.10.55:6443 --token 2lbtwj.gnpknhy7yow5jqkg \\ --discovery-token-ca-cert-hash sha256:e5be5c6d9564bed4319dbbd872b105c401e0aed482bc131b6a4759ab5a279bcf クラスタの確認 1 kubectl get nodes -o wide 1 2 3 4 5 mao@cilium-control-plane-01:~$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME cilium-control-plane-01 Ready control-plane 2m33s v1.31.0 192.168.10.55 \u0026lt;none\u0026gt; Ubuntu 24.04.1 LTS 6.8.0-41-generic cri-o://1.30.5 cilium-woker-node-01 Ready \u0026lt;none\u0026gt; 68s v1.31.0 192.168.10.56 \u0026lt;none\u0026gt; Ubuntu 24.04.1 LTS 6.8.0-41-generic cri-o://1.30.5 mao@cilium-control-plane-01:~$ Helmのインストール 下記のコマンドを実行してHelmをインストールします\nhttps://github.com/helm/helm/releases 1 2 3 4 wget https://get.helm.sh/helm-v3.15.4-linux-amd64.tar.gz tar -zxvf helm-v3.15.4-linux-amd64.tar.gz sudo mv linux-amd64/helm /usr/local/bin/helm helm version インストール完了（バージョン確認）\n1 2 mao@cilium-control-plane-01:~$ helm version version.BuildInfo{Version:\u0026#34;v3.15.4\u0026#34;, GitCommit:\u0026#34;fa9efb07d9d8debbb4306d72af76a383895aa8c4\u0026#34;, GitTreeState:\u0026#34;clean\u0026#34;, GoVersion:\u0026#34;go1.22.6\u0026#34;} Helmでデプロイしたリソースのアンインストール方法\n1 helm uninstall release_name -n release_namespace Ciliumのデプロイ リポジトリを追加します 1 helm repo add cilium https://helm.cilium.io/ 追加されたか確認をします\n1 helm repo list 1 2 3 mao@cilium-control-plane-01:~$ helm repo list NAME URL cilium https://helm.cilium.io/ Ciliumをインストールします 下記のファイルをダウンロードしてCIDRを書き換えます\n1784行目くらいにある デフォルト\u0026quot;10.0.0.0/8\u0026quot; 変更後\u0026quot;10.128.0.0/16\u0026quot; 1 wget https://raw.githubusercontent.com/cilium/cilium/v1.16.1/install/kubernetes/cilium/values.yaml \u0026ldquo;vakues.yaml\u0026quot;を指定してインストールする 1 helm install cilium cilium/cilium --version 1.16.1 --namespace kube-system -f values.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 mao@cilium-control-plane-01:~$ helm install cilium cilium/cilium --version 1.16.1 --namespace kube-system -f values.yaml NAME: cilium LAST DEPLOYED: Wed Sep 4 11:53:51 2024 NAMESPACE: kube-system STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: You have successfully installed Cilium with Hubble. Your release version is 1.16.1. For any further help, visit https://docs.cilium.io/en/v1.16/gettinghelp mao@cilium-control-plane-01:~$ デプロイされているか確認をする\n1 kubectl get pod -A -o wide 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 mao@cilium-control-plane-01:~$ kubectl -n kube-system get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES cilium-6swqr 1/1 Running 0 80s 192.168.10.56 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; cilium-envoy-5rlzw 1/1 Running 0 79s 192.168.10.56 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; cilium-envoy-clh2j 1/1 Running 0 80s 192.168.10.55 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; cilium-operator-5c7867ccd5-ngjj8 1/1 Running 0 79s 192.168.10.55 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; cilium-operator-5c7867ccd5-qrsnp 1/1 Running 0 79s 192.168.10.56 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; cilium-z44kx 1/1 Running 0 79s 192.168.10.55 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; coredns-6f6b679f8f-66qm2 1/1 Running 0 64s 10.0.0.159 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; coredns-6f6b679f8f-wkpjz 1/1 Running 0 49s 10.0.0.177 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; etcd-cilium-control-plane-01 1/1 Running 0 5m17s 192.168.10.55 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-apiserver-cilium-control-plane-01 1/1 Running 0 5m17s 192.168.10.55 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-controller-manager-cilium-control-plane-01 1/1 Running 0 5m17s 192.168.10.55 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-proxy-86htv 1/1 Running 0 5m12s 192.168.10.55 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-proxy-msrd4 1/1 Running 0 3m55s 192.168.10.56 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-scheduler-cilium-control-plane-01 1/1 Running 0 5m17s 192.168.10.55 cilium-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; mao@cilium-control-plane-01:~$ Ciliumのアンインストール方法 1 helm uninstall cilium -n kube-system クラスタの再構築（手順を間違えた場合） Contrpl-Planeで実行 1 2 kubectl drain cilium-woker-node-01 --ignore-daemonsets --delete-emptydir-data --force kubectl delete node cilium-woker-node-01 Woker-Nodeで実行 1 2 3 4 sudo kubeadm reset sudo ip link sudo ip link delete cilium_vxlan sudo ip link Contrpl-Planeで実行 1 2 3 4 5 6 7 sudo kubeadm reset sudo rm -rf $HOME/.kube sudo systemctl daemon-reload \u0026amp;\u0026amp; systemctl restart kubelet sudo systemctl restart cri-o sudo ip link sudo ip link delete cilium_vxlan sudo ip link Hubble-UIにアクセスできるようにする “vakues.yaml\u0026quot;を編集する\n1307行目、\u0026ldquo;hubble.relay.enabled\u0026rdquo; 1 2 - #enabled: false + enabled: true 1523行目、\u0026ldquo;hubble.ui.enabled\u0026rdquo; 1 2 - #enabled: false + enabled: true 1683行目、\u0026ldquo;hubble.ui.service.type\u0026rdquo; 1 2 - #type: ClusterIP + type: LoadBalancer デプロイする（アップグレードする）\n1 helm upgrade cilium cilium/cilium --version 1.16.1 --namespace kube-system -f values.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 mao@cilium-control-plane-01:~$ helm upgrade cilium cilium/cilium --version 1.16.1 --namespace kube-system -f values.yaml Error: UPGRADE FAILED: execution error at (cilium/templates/validate.yaml:4:7): Hubble UI requires .Values.hubble.relay.enabled=true mao@cilium-control-plane-01:~$ helm upgrade cilium cilium/cilium --version 1.16.1 --namespace kube-system -f values.yaml Release \u0026#34;cilium\u0026#34; has been upgraded. Happy Helming! NAME: cilium LAST DEPLOYED: Fri Sep 6 12:27:38 2024 NAMESPACE: kube-system STATUS: deployed REVISION: 2 TEST SUITE: None NOTES: You have successfully installed Cilium with Hubble Relay and Hubble UI. Your release version is 1.16.1. For any further help, visit https://docs.cilium.io/en/v1.16/gettinghelp mao@cilium-control-plane-01:~$ Hubble-UIのIPアドレスを確認する\n1 kubectl -n kube-system get service 1 2 3 4 5 6 7 mao@cilium-control-plane-01:~$ kubectl -n kube-system get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE hubble-peer ClusterIP 10.100.129.52 \u0026lt;none\u0026gt; 443/TCP 47h hubble-relay ClusterIP 10.101.171.5 \u0026lt;none\u0026gt; 80/TCP 2m35s hubble-ui LoadBalancer 10.97.255.102 192.168.10.60 80:30533/TCP 2m35s kube-dns ClusterIP 10.96.0.10 \u0026lt;none\u0026gt; 53/UDP,53/TCP,9153/TCP 47h mao@cilium-control-plane-01:~$ \u0026ldquo;EXTERNAL-IP\u0026quot;に記載されているIPアドレスにアクセスする\n\u0026ldquo;192.168.10.60\u0026rdquo; 参考URL HelmチャートでKubernetesにCiliumをインストール https://qiita.com/showchan33/items/f336d46af383d4c746d4 Cilium https://github.com/cilium/cilium/tree/v1.16.1/install/kubernetes/cilium https://docs.cilium.io/en/stable/network/kubernetes/concepts/ Installation using Helm https://docs.cilium.io/en/stable/installation/k8s-install-helm/ AWSのEC2インスタンスでKubernetesを作ってみる https://qiita.com/showchan33/items/02e4a5f02b08c08d7813 kubeadm+containerd+ciliumを用いてk8s構築し、hubbleの動作確認するまで試した https://qiita.com/fruscianteee/items/2b130eaa8418b183d515 EKS上にCiliumサービスメッシュを稼動させてみた! https://qiita.com/daitak/items/9749c3c6d9c489351ef6 ","date":"2024-09-16T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-cni-cilium/","title":"KubernetesのCNIにCiliumを使用して構築する"},{"content":"環境 Kubernetes 1.31.0 Control-Plane：1台 Woker-Node：3台+1台 cri-o v1.30.4 CNI(Container Networking Interface) Calico v3.28.0 Cilium v1.16.1 Rook-Ceph 1.15 参考URL https://rook.io/docs/rook/latest-release/Getting-Started/quickstart https://qiita.com/ohtsuka-shota/items/cc45d595f742f3625099 https://qiita.com/takuya_0301/items/5f6ad4fa197d0e4f9815 https://qiita.com/showchan33/items/f678665c5fddb3bf7551 https://qiita.com/t_ume/items/4ac37f746bf07146d5f9 https://zenn.dev/vampire_yuta/articles/367ee53b78d681 https://techstep.hatenablog.com/entry/2020/02/02/134702 CNIにCalicoを使用していると動作しない https://hidemium.hatenablog.com/entry/2023/01/21/083020 https://github.com/rook/rook/issues/7769 インストール インストール手順は公式ドキュメントを参考にする\nhttps://rook.io/docs/rook/latest-release/Getting-Started/quickstart マニフェストファイルをダウンロードする\n1 2 3 4 wget https://raw.githubusercontent.com/rook/rook/release-1.15/deploy/examples/crds.yaml wget https://raw.githubusercontent.com/rook/rook/release-1.15/deploy/examples/common.yaml wget https://raw.githubusercontent.com/rook/rook/release-1.15/deploy/examples/operator.yaml wget https://raw.githubusercontent.com/rook/rook/release-1.15/deploy/examples/cluster.yaml CNIがCaliicoの場合（上手く行かない） いくつかのPodが\u0026quot;Pending\u0026quot;になってしまう Rookをデプロイする 下記コマンドを実行して\u0026quot;Rook\u0026quot;をデプロイする\n1 kubectl create -f crds.yaml -f common.yaml -f operator.yaml デプロイされたか確認する\n1 kubectl -n rook-ceph get pod 1 2 3 4 mao@cilium-control-plane-01:~$ kubectl -n rook-ceph get pod NAME READY STATUS RESTARTS AGE rook-ceph-operator-b777cf696-d2v8d 1/1 Running 0 2m19s mao@cilium-control-plane-01:~$ 1 kubectl get all -n rook-ceph 1 2 3 4 5 6 7 8 9 10 mao@cilium-control-plane-01:~$ kubectl get all -n rook-ceph NAME READY STATUS RESTARTS AGE pod/rook-ceph-operator-b777cf696-d2v8d 1/1 Running 0 3m6s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/rook-ceph-operator 1/1 1 1 3m6s NAME DESIRED CURRENT READY AGE replicaset.apps/rook-ceph-operator-b777cf696 1 1 1 3m6s mao@cilium-control-plane-01:~$ Cephをデプロイする 1 2 kubectl create -f cluster.yaml kubectl apply -f cluster.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 mao@cri-o-control-plane-01:~$ kubectl -n rook-ceph get pod NAME READY STATUS RESTARTS AGE csi-cephfsplugin-frgl7 3/3 Running 0 33s csi-cephfsplugin-provisioner-76cc6d46cd-88s2t 0/6 ContainerCreating 0 33s csi-cephfsplugin-provisioner-76cc6d46cd-8p4wr 0/6 ContainerCreating 0 33s csi-cephfsplugin-rbjwz 3/3 Running 0 33s csi-cephfsplugin-rrc96 0/3 ContainerCreating 0 33s csi-rbdplugin-2l57x 3/3 Running 0 34s csi-rbdplugin-9mjr7 0/3 ContainerCreating 0 34s csi-rbdplugin-b64g2 3/3 Running 0 34s csi-rbdplugin-provisioner-cd76f59d6-m2dtd 0/6 ContainerCreating 0 33s csi-rbdplugin-provisioner-cd76f59d6-zrnlw 0/6 ContainerCreating 0 33s rook-ceph-csi-detect-version-2wfxz 0/1 PodInitializing 0 23s rook-ceph-mon-a-canary-644484764b-rlb5d 2/2 Running 0 36s rook-ceph-mon-b-canary-5cbb87459c-4jsrl 2/2 Running 0 36s rook-ceph-mon-c-canary-5c54488996-lcmsh 0/2 Pending 0 36s rook-ceph-operator-9b9dd876c-5xkpb 1/1 Running 0 3m29s mao@cri-o-control-plane-01:~$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 mao@cri-o-control-plane-01:~$ kubectl -n rook-ceph get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES csi-cephfsplugin-frgl7 3/3 Running 0 3m28s 192.168.10.56 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-cephfsplugin-provisioner-76cc6d46cd-88s2t 6/6 Running 0 3m28s 10.128.89.151 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-cephfsplugin-provisioner-76cc6d46cd-8p4wr 6/6 Running 1 (2m44s ago) 3m28s 10.128.42.210 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-cephfsplugin-rbjwz 3/3 Running 0 3m28s 192.168.10.58 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-cephfsplugin-rrc96 3/3 Running 2 (110s ago) 3m28s 192.168.10.57 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-2l57x 3/3 Running 0 3m29s 192.168.10.56 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-9mjr7 3/3 Running 2 (2m16s ago) 3m29s 192.168.10.57 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-b64g2 3/3 Running 0 3m29s 192.168.10.58 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-provisioner-cd76f59d6-m2dtd 6/6 Running 0 3m28s 10.128.89.150 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-provisioner-cd76f59d6-zrnlw 6/6 Running 1 (2m47s ago) 3m28s 10.128.42.209 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-csi-detect-version-2wfxz 0/1 Completed 0 3m18s 10.128.42.211 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-mon-a-5b99cfb698-n9df2 0/2 Pending 0 2m27s \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-operator-9b9dd876c-5xkpb 0/1 Completed 0 6m24s 10.128.42.206 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-operator-9b9dd876c-8jp86 0/1 Completed 0 2m20s 10.128.43.163 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-operator-9b9dd876c-xr24m 0/1 Pending 0 108s \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; mao@cri-o-control-plane-01:~$ 削除する 1 2 kubectl delete -f crds.yaml -f common.yaml -f operator.yaml kubectl delete -f cluster.yaml Kubernetes の Evicted な Pod を消してディスク領域を空ける https://zenn.dev/imksoo/articles/c3e27dc0392990 1 kubectl get pod -A --no-headers --field-selector status.phase!=Running | awk \u0026#39;{print \u0026#34;kubectl delete -n \u0026#34;$1\u0026#34; pod/\u0026#34;$2}\u0026#39; | sh CNIがCiliumの場合（上手く行く） Rookをデプロイする 下記コマンドを実行して\u0026quot;Rook\u0026quot;をデプロイする\n1 kubectl create -f crds.yaml -f common.yaml -f operator.yaml デプロイされたか確認する\n1 kubectl -n rook-ceph get pod 1 2 3 4 mao@cilium-control-plane-01:~$ kubectl -n rook-ceph get pod NAME READY STATUS RESTARTS AGE rook-ceph-operator-b777cf696-d2v8d 1/1 Running 0 2m19s mao@cilium-control-plane-01:~$ 1 kubectl get all -n rook-ceph 1 2 3 4 5 6 7 8 9 10 mao@cilium-control-plane-01:~$ kubectl get all -n rook-ceph NAME READY STATUS RESTARTS AGE pod/rook-ceph-operator-b777cf696-d2v8d 1/1 Running 0 3m6s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/rook-ceph-operator 1/1 1 1 3m6s NAME DESIRED CURRENT READY AGE replicaset.apps/rook-ceph-operator-b777cf696 1 1 1 3m6s mao@cilium-control-plane-01:~$ Cephをデプロイする \u0026ldquo;cluster.yaml\u0026quot;を編集する 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 storage: # cluster level storage configuration and selection #useAllNodes: true #useAllDevices: true useAllNodes: false useAllDevices: false #deviceFilter: config: # crushRoot: \u0026#34;custom-root\u0026#34; # specify a non-default root label for the CRUSH map # metadataDevice: \u0026#34;md0\u0026#34; # specify a non-rotational storage so ceph-volume will use it as block db device of bluestore. # databaseSizeMB: \u0026#34;1024\u0026#34; # uncomment if the disks are smaller than 100 GB # osdsPerDevice: \u0026#34;1\u0026#34; # this value can be overridden at the node or device level # encryptedDevice: \u0026#34;true\u0026#34; # the default value for this option is \u0026#34;false\u0026#34; # deviceClass: \u0026#34;myclass\u0026#34; # specify a device class for OSDs in the cluster allowDeviceClassUpdate: false # whether to allow changing the device class of an OSD after it is created allowOsdCrushWeightUpdate: false # whether to allow resizing the OSD crush weight after osd pvc is increased # Individual nodes and their config can be specified as well, but \u0026#39;useAllNodes\u0026#39; above must be set to false. Then, only the named # nodes below will be used as storage resources. Each node\u0026#39;s \u0026#39;name\u0026#39; field should match their \u0026#39;kubernetes.io/hostname\u0026#39; label. nodes: #- name: \u0026#34;192.168.10.56\u0026#34; - name: \u0026#34;cilium-woker-node-01\u0026#34; devices: - name: \u0026#34;sdb\u0026#34; #- name: \u0026#34;192.168.10.57\u0026#34; - name: \u0026#34;cilium-woker-node-01\u0026#34; devices: - name: \u0026#34;sdb\u0026#34; #- name: \u0026#34;192.168.10.58\u0026#34; - name: \u0026#34;cilium-woker-node-01\u0026#34; devices: - name: \u0026#34;sdb\u0026#34; # nodes: # - name: \u0026#34;172.17.4.201\u0026#34; # devices: # specific devices to use for storage can be specified for each node # - name: \u0026#34;sdb\u0026#34; # - name: \u0026#34;nvme01\u0026#34; # multiple osds can be created on high performance devices # config: # osdsPerDevice: \u0026#34;5\u0026#34; # - name: \u0026#34;/dev/disk/by-id/ata-ST4000DM004-XXXX\u0026#34; # devices can be specified using full udev paths # config: # configuration can be specified at the node level which overrides the cluster level config # - name: \u0026#34;172.17.4.301\u0026#34; # deviceFilter: \u0026#34;^sd.\u0026#34; # when onlyApplyOSDPlacement is false, will merge both placement.All() and placement.osd \u0026ldquo;nodes\u0026quot;の\u0026quot;name:\u0026ldquo;はIPアドレスだと動作しないのでNodeの名前を入力する（\u0026ldquo;cilium-woker-node-01\u0026quot;等） \u0026ldquo;storage.nodes.name.devices.name\u0026quot;は使用するストレージを選択する（この場合は\u0026quot;sdb\u0026quot;を使用している） https://rook.io/docs/rook/latest-release/CRDs/Cluster/host-cluster/#specific-nodes-and-devices デプロイする\n1 2 kubectl create -f cluster.yaml kubectl apply -f cluster.yaml 確認する\n1 kubectl get pod -n rook-ceph -o wide 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 mao@cilium-control-plane-01:~$ kubectl get pod -n rook-ceph -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES csi-cephfsplugin-gtfc5 3/3 Running 1 (66m ago) 67m 192.168.10.58 cilium-woker-node-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-cephfsplugin-provisioner-76cc6d46cd-78cns 6/6 Running 1 (66m ago) 67m 10.128.3.86 cilium-woker-node-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-cephfsplugin-provisioner-76cc6d46cd-jd26f 6/6 Running 0 67m 10.128.2.193 cilium-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-cephfsplugin-qvmmv 3/3 Running 0 67m 192.168.10.57 cilium-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-cephfsplugin-vj7s5 3/3 Running 1 (66m ago) 67m 192.168.10.56 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-mz4v6 3/3 Running 1 (67m ago) 67m 192.168.10.56 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-pggtk 3/3 Running 1 (67m ago) 67m 192.168.10.58 cilium-woker-node-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-provisioner-cd76f59d6-2tbhw 6/6 Running 1 (66m ago) 67m 10.128.3.138 cilium-woker-node-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-provisioner-cd76f59d6-ngmxj 6/6 Running 1 (66m ago) 67m 10.128.0.35 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-rbdplugin-zttwq 3/3 Running 0 67m 192.168.10.57 cilium-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-crashcollector-cilium-woker-node-01-8456599cf6-tgc64 1/1 Running 0 67m 10.128.0.132 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-crashcollector-cilium-woker-node-02-97bd68f69-rg25b 1/1 Running 0 67m 10.128.2.12 cilium-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-crashcollector-cilium-woker-node-03-6644b8c6d8-72j7c 1/1 Running 0 67m 10.128.3.134 cilium-woker-node-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-exporter-cilium-woker-node-01-5f4cf5f7f6-b74j2 1/1 Running 0 67m 10.128.0.133 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-exporter-cilium-woker-node-02-55674c8b97-t9rj5 1/1 Running 0 67m 10.128.2.218 cilium-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-exporter-cilium-woker-node-03-6cf4549856-mt9mb 1/1 Running 0 67m 10.128.3.210 cilium-woker-node-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-mgr-a-5cc85f554-vhz4f 3/3 Running 0 67m 10.128.0.188 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-mgr-b-6598b8cc89-7zh6m 3/3 Running 0 67m 10.128.2.130 cilium-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-mon-a-86bdc645fd-mmkk4 2/2 Running 0 70m 10.128.2.185 cilium-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-mon-b-669799cb8d-f9ktn 2/2 Running 0 69m 10.128.3.42 cilium-woker-node-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-mon-c-7d68b88dc9-k7jlv 2/2 Running 0 67m 10.128.0.131 cilium-woker-node-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; rook-ceph-operator-b777cf696-d2v8d 1/1 Running 0 78m 10.128.2.155 cilium-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; mao@cilium-control-plane-01:~$ 全て\u0026quot;Running\u0026quot;になっているので問題なくデプロイできている Web UI (Dashboard) LoadBalancer MetalLBをインストールする LoadBalancerデプロイ 参考URL\nhttps://rook.io/docs/rook/latest-release/Storage-Configuration/Monitoring/ceph-dashboard/#load-balancer https://github.com/rook/rook/blob/release-1.15/deploy/examples/dashboard-loadbalancer.yaml マニフェストファイルをダウンロードする\n1 wget https://raw.githubusercontent.com/rook/rook/release-1.15/deploy/examples/dashboard-loadbalancer.yaml デプロイする\n1 kubectl create -f dashboard-loadbalancer.yaml デプロイされたダッシュボードのIPアドレスを確認する\n1 kubectl -n rook-ceph get service 1 2 3 4 5 6 7 8 9 10 mao@cilium-control-plane-01:~$ kubectl -n rook-ceph get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE rook-ceph-exporter ClusterIP 10.108.226.114 \u0026lt;none\u0026gt; 9926/TCP 93m rook-ceph-mgr ClusterIP 10.98.27.89 \u0026lt;none\u0026gt; 9283/TCP 93m rook-ceph-mgr-dashboard ClusterIP 10.102.164.58 \u0026lt;none\u0026gt; 8443/TCP 93m rook-ceph-mgr-dashboard-loadbalancer LoadBalancer 10.98.225.42 192.168.10.59 8443:32740/TCP 2m8s rook-ceph-mon-a ClusterIP 10.105.48.50 \u0026lt;none\u0026gt; 6789/TCP,3300/TCP 97m rook-ceph-mon-b ClusterIP 10.97.27.164 \u0026lt;none\u0026gt; 6789/TCP,3300/TCP 96m rook-ceph-mon-c ClusterIP 10.102.18.161 \u0026lt;none\u0026gt; 6789/TCP,3300/TCP 94m mao@cilium-control-plane-01:~$ 下記のIPアドレスにアクセスする\n1 https://192.168.10.59:8443/ ダッシュボードのログイン情報の確認方法 パスワードの確認方法 1 kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath=\u0026#34;{[\u0026#39;data\u0026#39;][\u0026#39;password\u0026#39;]}\u0026#34; | base64 --decode \u0026amp;\u0026amp; echo 画面にアクセスして入力する\nadmin Fd)q*/TcDC.x-YaM+t1X ","date":"2024-09-15T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-rook-ceph/","title":"KubernetesにRook-Cephをインストールする"},{"content":"環境 Kubernetes 1.31.0 Control-Plane：1台 Woker-Node：3台+1台 cri-o v1.30.4 Longhorn 1.7.0 参考URL インストール手順\nhttps://longhorn.io/docs/1.7.0/deploy/install/install-with-kubectl/ インストール要件\nhttps://longhorn.io/docs/1.7.0/deploy/install/#installation-requirements https://longhorn.io/docs/1.7.0/advanced-resources/longhornctl/install-longhornctl/ Dashboardも表示できるようにする\nhttps://longhorn.io/docs/1.7.0/deploy/accessing-the-ui/ インストール手順 マニフェストファイルをダウンロードする\n1 wget https://raw.githubusercontent.com/longhorn/longhorn/v1.7.0/deploy/longhorn.yaml 4724行目あたり Dashboardを外部からアクセスできるように\u0026quot;type: LoadBalancer\u0026quot;する 1 2 - type: ClusterIP + type: LoadBalancer デプロイする\n1 2 kubectl apply -f longhorn.yaml kubectl delete -f longhorn.yaml インストールの進行状況を確認する\n1 2 3 kubectl get pods \\ --namespace longhorn-system \\ --watch デプロイができているか確認する\n1 2 kubectl -n longhorn-system get pod kubectl -n longhorn-system get svc Podがすぐ\u0026quot;Evicted\u0026quot;になってしまう（解決済み） 試したこと（どれも解決しなかった）\nEvictedになっているPodを削除した→変化なし、Podが再生成されるがEvictedになる メモリを16GBに増やす→変化なし Woker-Nodeを追加した→変化なし マニフェストファイル上の\u0026quot;replicas\u0026quot;の数を変更した→変化なし 解決した方法\nDashboard上で確認した際に\u0026quot;down\u0026quot;となっており、Evictedが表示されているPodがあるNodeのストレージ容量がぎりぎりになっていたため容量を増やした ストレージの容量を元々は10GBだったが20GBに拡張したら問題なく\u0026quot;Running\u0026quot;になり問題なく動作した \u0026ldquo;Evicted\u0026rdquo;,\u0026ldquo;ContainerStatusUnknown\u0026rdquo;,\u0026ldquo;Error\u0026quot;になってしまう場合はNodeのストレージの容量不足でなっていた 1 2 3 4 5 6 7 mao@cri-o-control-plane-01:~$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME cri-o-control-plane-01 Ready control-plane 4d17h v1.31.0 192.168.10.55 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-41-generic cri-o://1.30.4 cri-o-woker-node-02 Ready \u0026lt;none\u0026gt; 4d17h v1.31.0 192.168.10.58 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-41-generic cri-o://1.30.4 crio-woker-node-04 Ready \u0026lt;none\u0026gt; 87m v1.31.0 192.168.10.59 \u0026lt;none\u0026gt; Ubuntu 24.04.1 LTS 6.8.0-41-generic cri-o://1.30.4 k8s-crio-woker-node Ready \u0026lt;none\u0026gt; 46m v1.31.0 192.168.10.57 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-41-generic cri-o://1.30.4 mao-cri-o-worker-node Ready \u0026lt;none\u0026gt; 4d17h v1.31.0 192.168.10.56 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-41-generic cri-o://1.30.4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 mao@cri-o-control-plane-01:~$ kubectl -n longhorn-system get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES csi-attacher-d7d65647-7pmv5 1/1 Running 3 (93m ago) 22h 10.128.89.161 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-attacher-d7d65647-7sgfb 1/1 Running 2 (93m ago) 22h 10.128.89.164 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-attacher-d7d65647-xsrqw 1/1 Running 1 22h 10.128.43.177 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-provisioner-c4bc78768-8nnmq 1/1 Running 2 22h 10.128.43.176 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-provisioner-c4bc78768-bfvkv 1/1 Running 2 (93m ago) 22h 10.128.89.166 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-provisioner-c4bc78768-j9mvn 1/1 Running 3 (93m ago) 22h 10.128.89.169 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-resizer-5d979bd54f-mm2nv 1/1 Running 2 (93m ago) 22h 10.128.89.171 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-resizer-5d979bd54f-r9v2l 1/1 Running 1 22h 10.128.43.171 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-resizer-5d979bd54f-zsvsp 1/1 Running 2 (93m ago) 22h 10.128.89.160 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-snapshotter-6967b768c5-5k7bw 1/1 Running 2 (93m ago) 22h 10.128.89.168 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-snapshotter-6967b768c5-b8rhc 1/1 Running 1 22h 10.128.43.172 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-snapshotter-6967b768c5-bgndk 1/1 Running 2 (93m ago) 22h 10.128.89.170 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; engine-image-ei-04c05bf8-5scw8 1/1 Running 0 90m 10.85.0.3 crio-woker-node-04 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; engine-image-ei-04c05bf8-cwv4s 0/1 Error 0 50s 10.128.42.209 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; engine-image-ei-04c05bf8-js9cs 1/1 Running 1 22h 10.128.89.167 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; engine-image-ei-04c05bf8-qcsrv 1/1 Running 1 22h 10.128.43.181 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; instance-manager-0ecb4b94c60d35839b99dd2d623388ed 0/1 Evicted 0 31s \u0026lt;none\u0026gt; k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; instance-manager-aab3d619b1ba92d19591bd1d921a1146 1/1 Running 0 86m 10.128.158.65 crio-woker-node-04 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; instance-manager-ceb82c2dc60ec205037420596df48d66 1/1 Running 0 94m 10.128.89.173 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; instance-manager-d5708d70dcb6e03f00727b77e7699675 1/1 Running 0 94m 10.128.43.183 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-csi-plugin-9h5pz 3/3 Running 6 (93m ago) 22h 10.128.89.163 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-csi-plugin-cpt7t 3/3 Running 5 (94m ago) 22h 10.128.43.175 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-csi-plugin-d5mf9 0/3 Error 0 49s 10.128.42.210 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-csi-plugin-ncxxj 3/3 Running 5 (88m ago) 90m 10.85.0.2 crio-woker-node-04 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-driver-deployer-588c6bb4c8-6ldxd 1/1 Running 1 22h 10.128.43.180 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-manager-fc8qn 0/2 ContainerStatusUnknown 1 53s 10.128.42.208 k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-manager-lx4n2 2/2 Running 2 22h 10.128.89.162 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-manager-qwbxr 2/2 Running 3 (87m ago) 90m 10.85.0.4 crio-woker-node-04 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-manager-vkfm9 2/2 Running 2 22h 10.128.43.178 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-ui-5cb7d58f9c-44sr7 1/1 Running 2 (94m ago) 22h 10.128.89.172 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-ui-5cb7d58f9c-6jczn 1/1 Running 1 21h 10.128.43.174 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; mao@cri-o-control-plane-01:~$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 mao@cri-o-control-plane-01:~$ kubectl -n longhorn-system get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES csi-attacher-d7d65647-7pmv5 1/1 Running 3 (92m ago) 22h 10.128.89.161 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-attacher-d7d65647-7sgfb 1/1 Running 2 (92m ago) 22h 10.128.89.164 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-attacher-d7d65647-xsrqw 1/1 Running 1 22h 10.128.43.177 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-provisioner-c4bc78768-8nnmq 1/1 Running 2 22h 10.128.43.176 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-provisioner-c4bc78768-bfvkv 1/1 Running 2 (92m ago) 22h 10.128.89.166 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-provisioner-c4bc78768-j9mvn 1/1 Running 3 (92m ago) 22h 10.128.89.169 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-resizer-5d979bd54f-mm2nv 1/1 Running 2 (92m ago) 22h 10.128.89.171 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-resizer-5d979bd54f-r9v2l 1/1 Running 1 22h 10.128.43.171 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-resizer-5d979bd54f-zsvsp 1/1 Running 2 (92m ago) 22h 10.128.89.160 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-snapshotter-6967b768c5-5k7bw 1/1 Running 2 (92m ago) 22h 10.128.89.168 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-snapshotter-6967b768c5-b8rhc 1/1 Running 1 22h 10.128.43.172 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; csi-snapshotter-6967b768c5-bgndk 1/1 Running 2 (92m ago) 22h 10.128.89.170 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; engine-image-ei-04c05bf8-5scw8 1/1 Running 0 88m 10.85.0.3 crio-woker-node-04 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; engine-image-ei-04c05bf8-85qqz 0/1 Error 0 15m \u0026lt;none\u0026gt; k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; engine-image-ei-04c05bf8-js9cs 1/1 Running 1 22h 10.128.89.167 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; engine-image-ei-04c05bf8-qcsrv 1/1 Running 1 22h 10.128.43.181 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; instance-manager-0ecb4b94c60d35839b99dd2d623388ed 0/1 ContainerStatusUnknown 1 15m \u0026lt;none\u0026gt; k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; instance-manager-aab3d619b1ba92d19591bd1d921a1146 1/1 Running 0 85m 10.128.158.65 crio-woker-node-04 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; instance-manager-ceb82c2dc60ec205037420596df48d66 1/1 Running 0 92m 10.128.89.173 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; instance-manager-d5708d70dcb6e03f00727b77e7699675 1/1 Running 0 92m 10.128.43.183 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-csi-plugin-9h5pz 3/3 Running 6 (92m ago) 22h 10.128.89.163 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-csi-plugin-cpt7t 3/3 Running 5 (92m ago) 22h 10.128.43.175 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-csi-plugin-ncxxj 3/3 Running 5 (87m ago) 88m 10.85.0.2 crio-woker-node-04 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-csi-plugin-ndgbb 0/3 Error 1 15m \u0026lt;none\u0026gt; k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-driver-deployer-588c6bb4c8-6ldxd 1/1 Running 1 22h 10.128.43.180 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-manager-842xw 0/2 ContainerStatusUnknown 2 15m \u0026lt;none\u0026gt; k8s-crio-woker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-manager-lx4n2 2/2 Running 2 22h 10.128.89.162 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-manager-qwbxr 2/2 Running 3 (86m ago) 88m 10.85.0.4 crio-woker-node-04 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-manager-vkfm9 2/2 Running 2 22h 10.128.43.178 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-ui-5cb7d58f9c-44sr7 1/1 Running 2 (92m ago) 22h 10.128.89.172 mao-cri-o-worker-node \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; longhorn-ui-5cb7d58f9c-6jczn 1/1 Running 1 21h 10.128.43.174 cri-o-woker-node-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; mao@cri-o-control-plane-01:~$ Dashboardを確認する LoadBalancerを指定したのでIPアドレスが割り当てられています 下記コマンドで確認します 1 kubectl -n longhorn-system get service 1 2 3 4 5 6 7 8 mao@cri-o-control-plane-01:~$ kubectl -n longhorn-system get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE longhorn-admission-webhook ClusterIP 10.109.17.201 \u0026lt;none\u0026gt; 9502/TCP 24h longhorn-backend ClusterIP 10.102.186.222 \u0026lt;none\u0026gt; 9500/TCP 24h longhorn-conversion-webhook ClusterIP 10.103.48.37 \u0026lt;none\u0026gt; 9501/TCP 24h longhorn-frontend LoadBalancer 10.99.231.247 192.168.10.60 80:31171/TCP 24h longhorn-recovery-backend ClusterIP 10.111.12.211 \u0026lt;none\u0026gt; 9503/TCP 24h mao@cri-o-control-plane-01:~$ \u0026ldquo;192.168.10.60\u0026quot;にアクセスするとDashboardが表示されます ","date":"2024-09-14T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-longhorn/","title":"KubernetesにLonghornをインストールする"},{"content":"環境 Kubernetes 1.31.0 cri-o v1.30.4 参考URL https://github.com/cri-o/packaging/blob/main/README.md https://github.com/cri-o/cri-o/releases cri-oのインストール リポジトリを追加してのインストールと、バイナリをインストールの2種類がある\nリポジトリからインストールする 1 sudo apt install software-properties-common curl リポジトリの追加\n1 CRIO_VERSION=v1.30 1 2 3 4 5 curl -fsSL https://pkgs.k8s.io/addons:/cri-o:/stable:/$CRIO_VERSION/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/cri-o-apt-keyring.gpg echo \u0026#34;deb [signed-by=/etc/apt/keyrings/cri-o-apt-keyring.gpg] https://pkgs.k8s.io/addons:/cri-o:/stable:/$CRIO_VERSION/deb/ /\u0026#34; | tee /etc/apt/sources.list.d/cri-o.list 1 2 3 4 5 curl -fsSL https://pkgs.k8s.io/addons:/cri-o:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/cri-o-apt-keyring.gpg echo \u0026#34;deb [signed-by=/etc/apt/keyrings/cri-o-apt-keyring.gpg] https://pkgs.k8s.io/addons:/cri-o:/stable:/v1.30/deb/ /\u0026#34; | sudo tee /etc/apt/sources.list.d/cri-o.list cri-oをインストールする\n1 2 3 4 5 6 sudo apt install cri-o sudo systemctl daemon-reload systemctl enable --now cri-o systemctl start cri-o systemctl status cri-o crio version アンインストール方法\n1 which crio バイナリをインストール 上記のリポジトリを追加してインストールか、このバイナリをインストールする方法のどちらでも問題なく動作する\nバージョンアップはリポジトリの追加のほうがしやすい\n1 2 wget https://storage.googleapis.com/cri-o/artifacts/cri-o.$ARCH.$REV.tar.gz wget https://storage.googleapis.com/cri-o/artifacts/cri-o.amd64.v1.30.4.tar.gz 1 2 3 tar -zxvf cri-o.amd64.v1.30.4.tar.gz cd cri-o sudo bash ./install 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 mao@k8s-crio-woker-node:~/cri-o$ sudo bash ./install + DESTDIR= + PREFIX=/usr/local + ETCDIR=/etc + LIBEXECDIR=/usr/libexec + LIBEXEC_CRIO_DIR=/usr/libexec/crio + ETC_CRIO_DIR=/etc/crio + CONTAINERS_DIR=/etc/containers + CONTAINERS_REGISTRIES_CONFD_DIR=/etc/containers/registries.conf.d + CNIDIR=/etc/cni/net.d + BINDIR=/usr/local/bin + MANDIR=/usr/local/share/man + OCIDIR=/usr/local/share/oci-umount/oci-umount.d + BASHINSTALLDIR=/usr/local/share/bash-completion/completions + FISHINSTALLDIR=/usr/local/share/fish/completions + ZSHINSTALLDIR=/usr/local/share/zsh/site-functions + OPT_CNI_BIN_DIR=/opt/cni/bin + dpkg -l + SYSCONFIGDIR=/etc/default + sed -i \u0026#39;s;sysconfig/crio;default/crio;g\u0026#39; etc/crio + source /etc/os-release ++ PRETTY_NAME=\u0026#39;Ubuntu 24.04 LTS\u0026#39; ++ NAME=Ubuntu ++ VERSION_ID=24.04 ++ VERSION=\u0026#39;24.04 LTS (Noble Numbat)\u0026#39; ++ VERSION_CODENAME=noble ++ ID=ubuntu ++ ID_LIKE=debian ++ HOME_URL=https://www.ubuntu.com/ ++ SUPPORT_URL=https://help.ubuntu.com/ ++ BUG_REPORT_URL=https://bugs.launchpad.net/ubuntu/ ++ PRIVACY_POLICY_URL=https://www.ubuntu.com/legal/terms-and-policies/privacy-policy ++ UBUNTU_CODENAME=noble ++ LOGO=ubuntu-logo + [[ ubuntu == \\f\\e\\d\\o\\r\\a ]] + [[ ubuntu == \\r\\h\\c\\o\\s ]] + SYSTEMDDIR=/usr/local/lib/systemd/system + SELINUX= + selinuxenabled + ARCH=amd64 + install -d -m 755 /etc/cni/net.d + install -D -m 755 -t /opt/cni/bin cni-plugins/LICENSE cni-plugins/README.md cni-plugins/bandwidth cni-plugins/bridge cni-plugins/dhcp cni-plugins/dummy cni-plugins/firewall cni-plugins/host-device cni-plugins/host-local cni-plugins/ipvlan cni-plugins/loopback cni-plugins/macvlan cni-plugins/portmap cni-plugins/ptp cni-plugins/sbr cni-plugins/static cni-plugins/tap cni-plugins/tuning cni-plugins/vlan cni-plugins/vrf + install -D -m 644 -t /etc/cni/net.d contrib/11-crio-ipv4-bridge.conflist + install -d -m 755 /usr/libexec/crio + install -D -m 755 -t /usr/libexec/crio bin/conmon + install -D -m 755 -t /usr/libexec/crio bin/conmonrs + install -D -m 755 -t /usr/libexec/crio bin/crun + install -D -m 755 -t /usr/libexec/crio bin/runc + install -d -m 755 /usr/local/share/bash-completion/completions + install -d -m 755 /usr/local/share/fish/completions + install -d -m 755 /usr/local/share/zsh/site-functions + install -d -m 755 /etc/containers/registries.conf.d + install -D -m 755 -t /usr/local/bin bin/crio + install -D -m 755 -t /usr/local/bin bin/pinns + install -D -m 755 -t /usr/local/bin bin/crictl + install -D -m 644 -t /etc etc/crictl.yaml + install -D -m 644 -t /usr/local/share/oci-umount/oci-umount.d etc/crio-umount.conf + install -D -m 644 -t /etc/default etc/crio + install -D -m 644 -t /etc/crio contrib/policy.json + install -D -m 644 -t /etc/crio/crio.conf.d etc/10-crio.conf + install -D -m 644 -t /usr/local/share/man/man5 man/crio.conf.5 + install -D -m 644 -t /usr/local/share/man/man5 man/crio.conf.d.5 + install -D -m 644 -t /usr/local/share/man/man8 man/crio.8 + install -D -m 644 -t /usr/local/share/bash-completion/completions completions/bash/crio + install -D -m 644 -t /usr/local/share/fish/completions completions/fish/crio.fish + install -D -m 644 -t /usr/local/share/zsh/site-functions completions/zsh/_crio + install -D -m 644 -t /usr/local/lib/systemd/system contrib/crio.service + install -D -m 644 -t /etc/containers/registries.conf.d contrib/registries.conf + sed -i \u0026#39;s;/usr/bin;/usr/local/bin;g\u0026#39; /etc/crio/crio.conf.d/10-crio.conf + sed -i \u0026#39;s;/usr/libexec;/usr/libexec;g\u0026#39; /etc/crio/crio.conf.d/10-crio.conf + sed -i \u0026#39;s;/etc/crio;/etc/crio;g\u0026#39; /etc/crio/crio.conf.d/10-crio.conf + \u0026#39;[\u0026#39; -n \u0026#39;\u0026#39; \u0026#39;]\u0026#39; mao@k8s-crio-woker-node:~/cri-o$ 1 2 3 4 sudo systemctl daemon-reload sudo systemctl enable --now cri-o sudo systemctl status cri-o crio version 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 mao@k8s-crio-woker-node:~/cri-o$ sudo systemctl daemon-reload mao@k8s-crio-woker-node:~/cri-o$ sudo systemctl enable --now crio Created symlink /etc/systemd/system/cri-o.service → /usr/local/lib/systemd/system/crio.service. Created symlink /etc/systemd/system/multi-user.target.wants/crio.service → /usr/local/lib/systemd/system/crio.service. mao@k8s-crio-woker-node:~/cri-o$ crio version INFO[2024-08-23 23:59:44.088388712Z] Starting CRI-O, version: 1.30.4, git: dbc00ffd41a487c847158032193b6dca9b49e821(clean) Version: 1.30.4 GitCommit: dbc00ffd41a487c847158032193b6dca9b49e821 GitCommitDate: 2024-08-01T06:57:46Z GitTreeState: clean BuildDate: 1970-01-01T00:00:00Z GoVersion: go1.22.0 Compiler: gc Platform: linux/amd64 Linkmode: static BuildTags: static netgo osusergo exclude_graphdriver_btrfs exclude_graphdriver_devicemapper seccomp apparmor selinux LDFlags: unknown SeccompEnabled: true AppArmorEnabled: false mao@k8s-crio-woker-node:~/cri-o$ nanoをインストール 1 sudo apt install nano スワップメモリとIPアドレスの設定 スワップをオフにする\n1 2 3 sudo swapoff -a sudo nano /etc/fstab free -h IPアドレスを固定する\n1 2 3 4 5 6 ip link ip address sudo cp 99-config.yaml /etc/netplan/ sudo netplan apply try --timeout 10 sudo netplan apply sudo chmod 600 /etc/netplan/99-config.yaml カーネルパラメーター設定 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 cat \u0026lt;\u0026lt;EOF | sudo tee /etc/modules-load.d/k8s.conf overlay br_netfilter EOF sudo modprobe overlay sudo modprobe br_netfilter # この構成に必要なカーネルパラメーター、再起動しても値は永続します cat \u0026lt;\u0026lt;EOF | sudo tee /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 EOF # 再起動せずにカーネルパラメーターを適用 sudo sysctl --system 1 2 3 lsmod | grep br_netfilter lsmod | grep overlay sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward Cgroupfsの設定 1 stat -fc %T /sys/fs/cgroup/ 1 sudo crio config default | sudo tee /etc/crio/crio.conf 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 mao@mao-cri-o:~$ sudo crio config default | sudo tee /etc/crio/crio.conf INFO[2024-08-23 12:32:11.597692695Z] Starting CRI-O, version: 1.30.4, git: dbc00ffd41a487c847158032193b6dca9b49e821(clean) INFO Using default capabilities: CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FSETID, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_SETPCAP, CAP_NET_BIND_SERVICE, CAP_KILL # The CRI-O configuration file specifies all of the available configuration # options and command-line flags for the crio(8) OCI Kubernetes Container Runtime # daemon, but in a TOML format that can be more easily modified and versioned. # # Please refer to crio.conf(5) for details of all configuration options. # CRI-O supports partial configuration reload during runtime, which can be # done by sending SIGHUP to the running process. Currently supported options # are explicitly mentioned with: \u0026#39;This option supports live configuration # reload\u0026#39;. # CRI-O reads its storage defaults from the containers-storage.conf(5) file # located at /etc/containers/storage.conf. Modify this storage configuration if # you want to change the system\u0026#39;s defaults. If you want to modify storage just # for CRI-O, you can change the storage configuration options here. [crio] # Path to the \u0026#34;root directory\u0026#34;. CRI-O stores all of its data, including # containers images, in this directory. # root = \u0026#34;/var/lib/containers/storage\u0026#34; # Path to the \u0026#34;run directory\u0026#34;. CRI-O stores all of its state in this directory. # runroot = \u0026#34;/run/containers/storage\u0026#34; # Path to the \u0026#34;imagestore\u0026#34;. If CRI-O stores all of its images in this directory differently than Root. # imagestore = \u0026#34;\u0026#34; # Storage driver used to manage the storage of images and containers. Please # refer to containers-storage.conf(5) to see all available storage drivers. # storage_driver = \u0026#34;\u0026#34; # List to pass options to the storage driver. Please refer to # containers-storage.conf(5) to see all available storage options. # storage_option = [ # ] # The default log directory where all logs will go unless directly specified by # the kubelet. The log directory specified must be an absolute directory. # log_dir = \u0026#34;/var/log/crio/pods\u0026#34; # Location for CRI-O to lay down the temporary version file. # It is used to check if crio wipe should wipe containers, which should # always happen on a node reboot # version_file = \u0026#34;/var/run/crio/version\u0026#34; # Location for CRI-O to lay down the persistent version file. # It is used to check if crio wipe should wipe images, which should # only happen when CRI-O has been upgraded # version_file_persist = \u0026#34;\u0026#34; # InternalWipe is whether CRI-O should wipe containers and images after a reboot when the server starts. # If set to false, one must use the external command \u0026#39;crio wipe\u0026#39; to wipe the containers and images in these situations. # internal_wipe = true # InternalRepair is whether CRI-O should check if the container and image storage was corrupted after a sudden restart. # If it was, CRI-O also attempts to repair the storage. # internal_repair = false # Location for CRI-O to lay down the clean shutdown file. # It is used to check whether crio had time to sync before shutting down. # If not found, crio wipe will clear the storage directory. # clean_shutdown_file = \u0026#34;/var/lib/crio/clean.shutdown\u0026#34; # The crio.api table contains settings for the kubelet/gRPC interface. [crio.api] # Path to AF_LOCAL socket on which CRI-O will listen. # listen = \u0026#34;/var/run/crio/crio.sock\u0026#34; # IP address on which the stream server will listen. # stream_address = \u0026#34;127.0.0.1\u0026#34; # The port on which the stream server will listen. If the port is set to \u0026#34;0\u0026#34;, then # CRI-O will allocate a random free port number. # stream_port = \u0026#34;0\u0026#34; # Enable encrypted TLS transport of the stream server. # stream_enable_tls = false # Length of time until open streams terminate due to lack of activity # stream_idle_timeout = \u0026#34;\u0026#34; # Path to the x509 certificate file used to serve the encrypted stream. This # file can change, and CRI-O will automatically pick up the changes within 5 # minutes. # stream_tls_cert = \u0026#34;\u0026#34; # Path to the key file used to serve the encrypted stream. This file can # change and CRI-O will automatically pick up the changes within 5 minutes. # stream_tls_key = \u0026#34;\u0026#34; # Path to the x509 CA(s) file used to verify and authenticate client # communication with the encrypted stream. This file can change and CRI-O will # automatically pick up the changes within 5 minutes. # stream_tls_ca = \u0026#34;\u0026#34; # Maximum grpc send message size in bytes. If not set or \u0026lt;=0, then CRI-O will default to 80 * 1024 * 1024. # grpc_max_send_msg_size = 83886080 # Maximum grpc receive message size. If not set or \u0026lt;= 0, then CRI-O will default to 80 * 1024 * 1024. # grpc_max_recv_msg_size = 83886080 # The crio.runtime table contains settings pertaining to the OCI runtime used # and options for how to set up and manage the OCI runtime. [crio.runtime] # A list of ulimits to be set in containers by default, specified as # \u0026#34;\u0026lt;ulimit name\u0026gt;=\u0026lt;soft limit\u0026gt;:\u0026lt;hard limit\u0026gt;\u0026#34;, for example: # \u0026#34;nofile=1024:2048\u0026#34; # If nothing is set here, settings will be inherited from the CRI-O daemon # default_ulimits = [ # ] # If true, the runtime will not use pivot_root, but instead use MS_MOVE. # no_pivot = false # decryption_keys_path is the path where the keys required for # image decryption are stored. This option supports live configuration reload. # decryption_keys_path = \u0026#34;/etc/crio/keys/\u0026#34; # Path to the conmon binary, used for monitoring the OCI runtime. # Will be searched for using $PATH if empty. # This option is currently deprecated, and will be replaced with RuntimeHandler.MonitorEnv. # conmon = \u0026#34;\u0026#34; # Cgroup setting for conmon # This option is currently deprecated, and will be replaced with RuntimeHandler.MonitorCgroup. # conmon_cgroup = \u0026#34;\u0026#34; # Environment variable list for the conmon process, used for passing necessary # environment variables to conmon or the runtime. # This option is currently deprecated, and will be replaced with RuntimeHandler.MonitorEnv. # conmon_env = [ # ] # Additional environment variables to set for all the # containers. These are overridden if set in the # container image spec or in the container runtime configuration. # default_env = [ # ] # If true, SELinux will be used for pod separation on the host. # This option is deprecated, and be interpreted from whether SELinux is enabled on the host in the future. # selinux = false # Path to the seccomp.json profile which is used as the default seccomp profile # for the runtime. If not specified, then the internal default seccomp profile # will be used. This option supports live configuration reload. # seccomp_profile = \u0026#34;\u0026#34; # Used to change the name of the default AppArmor profile of CRI-O. The default # profile name is \u0026#34;crio-default\u0026#34;. This profile only takes effect if the user # does not specify a profile via the Kubernetes Pod\u0026#39;s metadata annotation. If # the profile is set to \u0026#34;unconfined\u0026#34;, then this equals to disabling AppArmor. # This option supports live configuration reload. # apparmor_profile = \u0026#34;crio-default\u0026#34; # Path to the blockio class configuration file for configuring # the cgroup blockio controller. # blockio_config_file = \u0026#34;\u0026#34; # Reload blockio-config-file and rescan blockio devices in the system before applying # blockio parameters. # blockio_reload = false # Used to change irqbalance service config file path which is used for configuring # irqbalance daemon. # irqbalance_config_file = \u0026#34;/etc/sysconfig/irqbalance\u0026#34; # irqbalance_config_restore_file allows to set a cpu mask CRI-O should # restore as irqbalance config at startup. Set to empty string to disable this flow entirely. # By default, CRI-O manages the irqbalance configuration to enable dynamic IRQ pinning. # irqbalance_config_restore_file = \u0026#34;/etc/sysconfig/orig_irq_banned_cpus\u0026#34; # Path to the RDT configuration file for configuring the resctrl pseudo-filesystem. # This option supports live configuration reload. # rdt_config_file = \u0026#34;\u0026#34; # Cgroup management implementation used for the runtime. # cgroup_manager = \u0026#34;systemd\u0026#34; # Specify whether the image pull must be performed in a separate cgroup. # separate_pull_cgroup = \u0026#34;\u0026#34; # List of default capabilities for containers. If it is empty or commented out, # only the capabilities defined in the containers json file by the user/kube # will be added. # default_capabilities = [ # \u0026#34;CHOWN\u0026#34;, # \u0026#34;DAC_OVERRIDE\u0026#34;, # \u0026#34;FSETID\u0026#34;, # \u0026#34;FOWNER\u0026#34;, # \u0026#34;SETGID\u0026#34;, # \u0026#34;SETUID\u0026#34;, # \u0026#34;SETPCAP\u0026#34;, # \u0026#34;NET_BIND_SERVICE\u0026#34;, # \u0026#34;KILL\u0026#34;, # ] # Add capabilities to the inheritable set, as well as the default group of permitted, bounding and effective. # If capabilities are expected to work for non-root users, this option should be set. # add_inheritable_capabilities = false # List of default sysctls. If it is empty or commented out, only the sysctls # defined in the container json file by the user/kube will be added. # default_sysctls = [ # ] # List of devices on the host that a # user can specify with the \u0026#34;io.kubernetes.cri-o.Devices\u0026#34; allowed annotation. # allowed_devices = [ # \u0026#34;/dev/fuse\u0026#34;, # ] # List of additional devices. specified as # \u0026#34;\u0026lt;device-on-host\u0026gt;:\u0026lt;device-on-container\u0026gt;:\u0026lt;permissions\u0026gt;\u0026#34;, for example: \u0026#34;--device=/dev/sdc:/dev/xvdc:rwm\u0026#34;. # If it is empty or commented out, only the devices # defined in the container json file by the user/kube will be added. # additional_devices = [ # ] # List of directories to scan for CDI Spec files. # cdi_spec_dirs = [ # \u0026#34;/etc/cdi\u0026#34;, # \u0026#34;/var/run/cdi\u0026#34;, # ] # Change the default behavior of setting container devices uid/gid from CRI\u0026#39;s # SecurityContext (RunAsUser/RunAsGroup) instead of taking host\u0026#39;s uid/gid. # Defaults to false. # device_ownership_from_security_context = false # Path to OCI hooks directories for automatically executed hooks. If one of the # directories does not exist, then CRI-O will automatically skip them. # hooks_dir = [ # \u0026#34;/usr/share/containers/oci/hooks.d\u0026#34;, # ] # Path to the file specifying the defaults mounts for each container. The # format of the config is /SRC:/DST, one mount per line. Notice that CRI-O reads # its default mounts from the following two files: # # 1) /etc/containers/mounts.conf (i.e., default_mounts_file): This is the # override file, where users can either add in their own default mounts, or # override the default mounts shipped with the package. # # 2) /usr/share/containers/mounts.conf: This is the default file read for # mounts. If you want CRI-O to read from a different, specific mounts file, # you can change the default_mounts_file. Note, if this is done, CRI-O will # only add mounts it finds in this file. # # default_mounts_file = \u0026#34;\u0026#34; # Maximum number of processes allowed in a container. # This option is deprecated. The Kubelet flag \u0026#39;--pod-pids-limit\u0026#39; should be used instead. # pids_limit = -1 # Maximum sized allowed for the container log file. Negative numbers indicate # that no size limit is imposed. If it is positive, it must be \u0026gt;= 8192 to # match/exceed conmon\u0026#39;s read buffer. The file is truncated and re-opened so the # limit is never exceeded. This option is deprecated. The Kubelet flag \u0026#39;--container-log-max-size\u0026#39; should be used instead. # log_size_max = -1 # Whether container output should be logged to journald in addition to the kubernetes log file # log_to_journald = false # Path to directory in which container exit files are written to by conmon. # container_exits_dir = \u0026#34;/var/run/crio/exits\u0026#34; # Path to directory for container attach sockets. # container_attach_socket_dir = \u0026#34;/var/run/crio\u0026#34; # The prefix to use for the source of the bind mounts. # bind_mount_prefix = \u0026#34;\u0026#34; # If set to true, all containers will run in read-only mode. # read_only = false # Changes the verbosity of the logs based on the level it is set to. Options # are fatal, panic, error, warn, info, debug and trace. This option supports # live configuration reload. # log_level = \u0026#34;info\u0026#34; # Filter the log messages by the provided regular expression. # This option supports live configuration reload. # log_filter = \u0026#34;\u0026#34; # The UID mappings for the user namespace of each container. A range is # specified in the form containerUID:HostUID:Size. Multiple ranges must be # separated by comma. # This option is deprecated, and will be replaced with Kubernetes user namespace support (KEP-127) in the future. # uid_mappings = \u0026#34;\u0026#34; # The GID mappings for the user namespace of each container. A range is # specified in the form containerGID:HostGID:Size. Multiple ranges must be # separated by comma. # This option is deprecated, and will be replaced with Kubernetes user namespace support (KEP-127) in the future. # gid_mappings = \u0026#34;\u0026#34; # If set, CRI-O will reject any attempt to map host UIDs below this value # into user namespaces. A negative value indicates that no minimum is set, # so specifying mappings will only be allowed for pods that run as UID 0. # This option is deprecated, and will be replaced with Kubernetes user namespace support (KEP-127) in the future. # minimum_mappable_uid = -1 # If set, CRI-O will reject any attempt to map host GIDs below this value # into user namespaces. A negative value indicates that no minimum is set, # so specifying mappings will only be allowed for pods that run as UID 0. # This option is deprecated, and will be replaced with Kubernetes user namespace support (KEP-127) in the future. # minimum_mappable_gid = -1 # The minimal amount of time in seconds to wait before issuing a timeout # regarding the proper termination of the container. The lowest possible # value is 30s, whereas lower values are not considered by CRI-O. # ctr_stop_timeout = 30 # drop_infra_ctr determines whether CRI-O drops the infra container # when a pod does not have a private PID namespace, and does not use # a kernel separating runtime (like kata). # It requires manage_ns_lifecycle to be true. # drop_infra_ctr = true # infra_ctr_cpuset determines what CPUs will be used to run infra containers. # You can use linux CPU list format to specify desired CPUs. # To get better isolation for guaranteed pods, set this parameter to be equal to kubelet reserved-cpus. # infra_ctr_cpuset = \u0026#34;\u0026#34; # shared_cpuset determines the CPU set which is allowed to be shared between guaranteed containers, # regardless of, and in addition to, the exclusiveness of their CPUs. # This field is optional and would not be used if not specified. # You can specify CPUs in the Linux CPU list format. # shared_cpuset = \u0026#34;\u0026#34; # The directory where the state of the managed namespaces gets tracked. # Only used when manage_ns_lifecycle is true. # namespaces_dir = \u0026#34;/var/run\u0026#34; # pinns_path is the path to find the pinns binary, which is needed to manage namespace lifecycle # pinns_path = \u0026#34;\u0026#34; # Globally enable/disable CRIU support which is necessary to # checkpoint and restore container or pods (even if CRIU is found in $PATH). # enable_criu_support = true # Enable/disable the generation of the container, # sandbox lifecycle events to be sent to the Kubelet to optimize the PLEG # enable_pod_events = false # default_runtime is the _name_ of the OCI runtime to be used as the default. # The name is matched against the runtimes map below. default_runtime = \u0026#34;crun\u0026#34; # A list of paths that, when absent from the host, # will cause a container creation to fail (as opposed to the current behavior being created as a directory). # This option is to protect from source locations whose existence as a directory could jeopardize the health of the node, and whose # creation as a file is not desired either. # An example is /etc/hostname, which will cause failures on reboot if it\u0026#39;s created as a directory, but often doesn\u0026#39;t exist because # the hostname is being managed dynamically. # absent_mount_sources_to_reject = [ # ] # The \u0026#34;crio.runtime.runtimes\u0026#34; table defines a list of OCI compatible runtimes. # The runtime to use is picked based on the runtime handler provided by the CRI. # If no runtime handler is provided, the \u0026#34;default_runtime\u0026#34; will be used. # Each entry in the table should follow the format: # # [crio.runtime.runtimes.runtime-handler] # runtime_path = \u0026#34;/path/to/the/executable\u0026#34; # runtime_type = \u0026#34;oci\u0026#34; # runtime_root = \u0026#34;/path/to/the/root\u0026#34; # monitor_path = \u0026#34;/path/to/container/monitor\u0026#34; # monitor_cgroup = \u0026#34;/cgroup/path\u0026#34; # monitor_exec_cgroup = \u0026#34;/cgroup/path\u0026#34; # monitor_env = [] # privileged_without_host_devices = false # allowed_annotations = [] # platform_runtime_paths = { \u0026#34;os/arch\u0026#34; = \u0026#34;/path/to/binary\u0026#34; } # Where: # - runtime-handler: Name used to identify the runtime. # - runtime_path (optional, string): Absolute path to the runtime executable in # the host filesystem. If omitted, the runtime-handler identifier should match # the runtime executable name, and the runtime executable should be placed # in $PATH. # - runtime_type (optional, string): Type of runtime, one of: \u0026#34;oci\u0026#34;, \u0026#34;vm\u0026#34;. If # omitted, an \u0026#34;oci\u0026#34; runtime is assumed. # - runtime_root (optional, string): Root directory for storage of containers # state. # - runtime_config_path (optional, string): the path for the runtime configuration # file. This can only be used with when using the VM runtime_type. # - privileged_without_host_devices (optional, bool): an option for restricting # host devices from being passed to privileged containers. # - allowed_annotations (optional, array of strings): an option for specifying # a list of experimental annotations that this runtime handler is allowed to process. # The currently recognized values are: # \u0026#34;io.kubernetes.cri-o.userns-mode\u0026#34; for configuring a user namespace for the pod. # \u0026#34;io.kubernetes.cri-o.cgroup2-mount-hierarchy-rw\u0026#34; for mounting cgroups writably when set to \u0026#34;true\u0026#34;. # \u0026#34;io.kubernetes.cri-o.Devices\u0026#34; for configuring devices for the pod. # \u0026#34;io.kubernetes.cri-o.ShmSize\u0026#34; for configuring the size of /dev/shm. # \u0026#34;io.kubernetes.cri-o.UnifiedCgroup.$CTR_NAME\u0026#34; for configuring the cgroup v2 unified block for a container. # \u0026#34;io.containers.trace-syscall\u0026#34; for tracing syscalls via the OCI seccomp BPF hook. # \u0026#34;io.kubernetes.cri-o.seccompNotifierAction\u0026#34; for enabling the seccomp notifier feature. # \u0026#34;io.kubernetes.cri-o.umask\u0026#34; for setting the umask for container init process. # \u0026#34;io.kubernetes.cri.rdt-class\u0026#34; for setting the RDT class of a container # \u0026#34;seccomp-profile.kubernetes.cri-o.io\u0026#34; for setting the seccomp profile for: # - a specific container by using: \u0026#34;seccomp-profile.kubernetes.cri-o.io/\u0026lt;CONTAINER_NAME\u0026gt;\u0026#34; # - a whole pod by using: \u0026#34;seccomp-profile.kubernetes.cri-o.io/POD\u0026#34; # Note that the annotation works on containers as well as on images. # For images, the plain annotation \u0026#34;seccomp-profile.kubernetes.cri-o.io\u0026#34; # can be used without the required \u0026#34;/POD\u0026#34; suffix or a container name. # \u0026#34;io.kubernetes.cri-o.DisableFIPS\u0026#34; for disabling FIPS mode in a Kubernetes pod within a FIPS-enabled cluster. # - monitor_path (optional, string): The path of the monitor binary. Replaces # deprecated option \u0026#34;conmon\u0026#34;. # - monitor_cgroup (optional, string): The cgroup the container monitor process will be put in. # Replaces deprecated option \u0026#34;conmon_cgroup\u0026#34;. # - monitor_exec_cgroup (optional, string): If set to \u0026#34;container\u0026#34;, indicates exec probes # should be moved to the container\u0026#39;s cgroup # - monitor_env (optional, array of strings): Environment variables to pass to the montior. # Replaces deprecated option \u0026#34;conmon_env\u0026#34;. # - platform_runtime_paths (optional, map): A mapping of platforms to the corresponding # runtime executable paths for the runtime handler. # - container_min_memory (optional, string): The minimum memory that must be set for a container. # This value can be used to override the currently set global value for a specific runtime. If not set, # a global default value of \u0026#34;12 MiB\u0026#34; will be used. # # Using the seccomp notifier feature: # # This feature can help you to debug seccomp related issues, for example if # blocked syscalls (permission denied errors) have negative impact on the workload. # # To be able to use this feature, configure a runtime which has the annotation # \u0026#34;io.kubernetes.cri-o.seccompNotifierAction\u0026#34; in the allowed_annotations array. # # It also requires at least runc 1.1.0 or crun 0.19 which support the notifier # feature. # # If everything is setup, CRI-O will modify chosen seccomp profiles for # containers if the annotation \u0026#34;io.kubernetes.cri-o.seccompNotifierAction\u0026#34; is # set on the Pod sandbox. CRI-O will then get notified if a container is using # a blocked syscall and then terminate the workload after a timeout of 5 # seconds if the value of \u0026#34;io.kubernetes.cri-o.seccompNotifierAction=stop\u0026#34;. # # This also means that multiple syscalls can be captured during that period, # while the timeout will get reset once a new syscall has been discovered. # # This also means that the Pods \u0026#34;restartPolicy\u0026#34; has to be set to \u0026#34;Never\u0026#34;, # otherwise the kubelet will restart the container immediately. # # Please be aware that CRI-O is not able to get notified if a syscall gets # blocked based on the seccomp defaultAction, which is a general runtime # limitation. [crio.runtime.runtimes.crun] runtime_path = \u0026#34;/usr/libexec/crio/crun\u0026#34; runtime_type = \u0026#34;\u0026#34; runtime_root = \u0026#34;/run/crun\u0026#34; runtime_config_path = \u0026#34;\u0026#34; container_min_memory = \u0026#34;\u0026#34; monitor_path = \u0026#34;/usr/libexec/crio/conmon\u0026#34; monitor_cgroup = \u0026#34;system.slice\u0026#34; monitor_exec_cgroup = \u0026#34;\u0026#34; allowed_annotations = [ \u0026#34;io.containers.trace-syscall\u0026#34;, ] privileged_without_host_devices = false [crio.runtime.runtimes.runc] runtime_path = \u0026#34;/usr/libexec/crio/runc\u0026#34; runtime_type = \u0026#34;\u0026#34; runtime_root = \u0026#34;/run/runc\u0026#34; runtime_config_path = \u0026#34;\u0026#34; container_min_memory = \u0026#34;\u0026#34; monitor_path = \u0026#34;/usr/libexec/crio/conmon\u0026#34; monitor_cgroup = \u0026#34;system.slice\u0026#34; monitor_exec_cgroup = \u0026#34;\u0026#34; privileged_without_host_devices = false # The workloads table defines ways to customize containers with different resources # that work based on annotations, rather than the CRI. # Note, the behavior of this table is EXPERIMENTAL and may change at any time. # Each workload, has a name, activation_annotation, annotation_prefix and set of resources it supports mutating. # The currently supported resources are \u0026#34;cpuperiod\u0026#34; \u0026#34;cpuquota\u0026#34;, \u0026#34;cpushares\u0026#34;, \u0026#34;cpulimit\u0026#34; and \u0026#34;cpuset\u0026#34;. The values for \u0026#34;cpuperiod\u0026#34; and \u0026#34;cpuquota\u0026#34; are denoted in microseconds. # The value for \u0026#34;cpulimit\u0026#34; is denoted in millicores, this value is used to calculate the \u0026#34;cpuquota\u0026#34; with the supplied \u0026#34;cpuperiod\u0026#34; or the default \u0026#34;cpuperiod\u0026#34;. # Note that the \u0026#34;cpulimit\u0026#34; field overrides the \u0026#34;cpuquota\u0026#34; value supplied in this configuration. # Each resource can have a default value specified, or be empty. # For a container to opt-into this workload, the pod should be configured with the annotation $activation_annotation (key only, value is ignored). # To customize per-container, an annotation of the form $annotation_prefix.$resource/$ctrName = \u0026#34;value\u0026#34; can be specified # signifying for that resource type to override the default value. # If the annotation_prefix is not present, every container in the pod will be given the default values. # Example: # [crio.runtime.workloads.workload-type] # activation_annotation = \u0026#34;io.crio/workload\u0026#34; # annotation_prefix = \u0026#34;io.crio.workload-type\u0026#34; # [crio.runtime.workloads.workload-type.resources] # cpuset = \u0026#34;0-1\u0026#34; # cpushares = \u0026#34;5\u0026#34; # cpuquota = \u0026#34;1000\u0026#34; # cpuperiod = \u0026#34;100000\u0026#34; # cpulimit = \u0026#34;35\u0026#34; # Where: # The workload name is workload-type. # To specify, the pod must have the \u0026#34;io.crio.workload\u0026#34; annotation (this is a precise string match). # This workload supports setting cpuset and cpu resources. # annotation_prefix is used to customize the different resources. # To configure the cpu shares a container gets in the example above, the pod would have to have the following annotation: # \u0026#34;io.crio.workload-type/$container_name = {\u0026#34;cpushares\u0026#34;: \u0026#34;value\u0026#34;}\u0026#34; # hostnetwork_disable_selinux determines whether # SELinux should be disabled within a pod when it is running in the host network namespace # Default value is set to true # hostnetwork_disable_selinux = true # disable_hostport_mapping determines whether to enable/disable # the container hostport mapping in CRI-O. # Default value is set to \u0026#39;false\u0026#39; # disable_hostport_mapping = false # timezone To set the timezone for a container in CRI-O. # If an empty string is provided, CRI-O retains its default behavior. Use \u0026#39;Local\u0026#39; to match the timezone of the host machine. # timezone = \u0026#34;\u0026#34; # The crio.image table contains settings pertaining to the management of OCI images. # # CRI-O reads its configured registries defaults from the system wide # containers-registries.conf(5) located in /etc/containers/registries.conf. If # you want to modify just CRI-O, you can change the registries configuration in # this file. Otherwise, leave insecure_registries and registries commented out to # use the system\u0026#39;s defaults from /etc/containers/registries.conf. [crio.image] # Default transport for pulling images from a remote container storage. # default_transport = \u0026#34;docker://\u0026#34; # The path to a file containing credentials necessary for pulling images from # secure registries. The file is similar to that of /var/lib/kubelet/config.json # global_auth_file = \u0026#34;\u0026#34; # The image used to instantiate infra containers. # This option supports live configuration reload. # pause_image = \u0026#34;registry.k8s.io/pause:3.9\u0026#34; # The path to a file containing credentials specific for pulling the pause_image from # above. The file is similar to that of /var/lib/kubelet/config.json # This option supports live configuration reload. # pause_image_auth_file = \u0026#34;\u0026#34; # The command to run to have a container stay in the paused state. # When explicitly set to \u0026#34;\u0026#34;, it will fallback to the entrypoint and command # specified in the pause image. When commented out, it will fallback to the # default: \u0026#34;/pause\u0026#34;. This option supports live configuration reload. # pause_command = \u0026#34;/pause\u0026#34; # List of images to be excluded from the kubelet\u0026#39;s garbage collection. # It allows specifying image names using either exact, glob, or keyword # patterns. Exact matches must match the entire name, glob matches can # have a wildcard * at the end, and keyword matches can have wildcards # on both ends. By default, this list includes the \u0026#34;pause\u0026#34; image if # configured by the user, which is used as a placeholder in Kubernetes pods. # pinned_images = [ # ] # Path to the file which decides what sort of policy we use when deciding # whether or not to trust an image that we\u0026#39;ve pulled. It is not recommended that # this option be used, as the default behavior of using the system-wide default # policy (i.e., /etc/containers/policy.json) is most often preferred. Please # refer to containers-policy.json(5) for more details. signature_policy = \u0026#34;/etc/crio/policy.json\u0026#34; # Root path for pod namespace-separated signature policies. # The final policy to be used on image pull will be \u0026lt;SIGNATURE_POLICY_DIR\u0026gt;/\u0026lt;NAMESPACE\u0026gt;.json. # If no pod namespace is being provided on image pull (via the sandbox config), # or the concatenated path is non existent, then the signature_policy or system # wide policy will be used as fallback. Must be an absolute path. # signature_policy_dir = \u0026#34;/etc/crio/policies\u0026#34; # List of registries to skip TLS verification for pulling images. Please # consider configuring the registries via /etc/containers/registries.conf before # changing them here. # insecure_registries = [ # ] # Controls how image volumes are handled. The valid values are mkdir, bind and # ignore; the latter will ignore volumes entirely. # image_volumes = \u0026#34;mkdir\u0026#34; # Temporary directory to use for storing big files # big_files_temporary_dir = \u0026#34;\u0026#34; # If true, CRI-O will automatically reload the mirror registry when # there is an update to the \u0026#39;registries.conf.d\u0026#39; directory. Default value is set to \u0026#39;false\u0026#39;. # auto_reload_registries = false # The crio.network table containers settings pertaining to the management of # CNI plugins. [crio.network] # The default CNI network name to be selected. If not set or \u0026#34;\u0026#34;, then # CRI-O will pick-up the first one found in network_dir. # cni_default_network = \u0026#34;\u0026#34; # Path to the directory where CNI configuration files are located. # network_dir = \u0026#34;/etc/cni/net.d/\u0026#34; # Paths to directories where CNI plugin binaries are located. # plugin_dirs = [ # \u0026#34;/opt/cni/bin/\u0026#34;, # ] # List of included pod metrics. # included_pod_metrics = [ # ] # A necessary configuration for Prometheus based metrics retrieval [crio.metrics] # Globally enable or disable metrics support. # enable_metrics = false # Specify enabled metrics collectors. # Per default all metrics are enabled. # It is possible, to prefix the metrics with \u0026#34;container_runtime_\u0026#34; and \u0026#34;crio_\u0026#34;. # For example, the metrics collector \u0026#34;operations\u0026#34; would be treated in the same # way as \u0026#34;crio_operations\u0026#34; and \u0026#34;container_runtime_crio_operations\u0026#34;. # metrics_collectors = [ # \u0026#34;image_pulls_layer_size\u0026#34;, # \u0026#34;containers_events_dropped_total\u0026#34;, # \u0026#34;containers_oom_total\u0026#34;, # \u0026#34;processes_defunct\u0026#34;, # \u0026#34;operations_total\u0026#34;, # \u0026#34;operations_latency_seconds\u0026#34;, # \u0026#34;operations_latency_seconds_total\u0026#34;, # \u0026#34;operations_errors_total\u0026#34;, # \u0026#34;image_pulls_bytes_total\u0026#34;, # \u0026#34;image_pulls_skipped_bytes_total\u0026#34;, # \u0026#34;image_pulls_failure_total\u0026#34;, # \u0026#34;image_pulls_success_total\u0026#34;, # \u0026#34;image_layer_reuse_total\u0026#34;, # \u0026#34;containers_oom_count_total\u0026#34;, # \u0026#34;containers_seccomp_notifier_count_total\u0026#34;, # \u0026#34;resources_stalled_at_stage\u0026#34;, # ] # The IP address or hostname on which the metrics server will listen. # metrics_host = \u0026#34;127.0.0.1\u0026#34; # The port on which the metrics server will listen. # metrics_port = 9090 # Local socket path to bind the metrics server to # metrics_socket = \u0026#34;\u0026#34; # The certificate for the secure metrics server. # If the certificate is not available on disk, then CRI-O will generate a # self-signed one. CRI-O also watches for changes of this path and reloads the # certificate on any modification event. # metrics_cert = \u0026#34;\u0026#34; # The certificate key for the secure metrics server. # Behaves in the same way as the metrics_cert. # metrics_key = \u0026#34;\u0026#34; # A necessary configuration for OpenTelemetry trace data exporting [crio.tracing] # Globally enable or disable exporting OpenTelemetry traces. # enable_tracing = false # Address on which the gRPC trace collector listens on. # tracing_endpoint = \u0026#34;0.0.0.0:4317\u0026#34; # Number of samples to collect per million spans. Set to 1000000 to always sample. # tracing_sampling_rate_per_million = 0 # CRI-O NRI configuration. [crio.nri] # Globally enable or disable NRI. # enable_nri = true # NRI socket to listen on. # nri_listen = \u0026#34;/var/run/nri/nri.sock\u0026#34; # NRI plugin directory to use. # nri_plugin_dir = \u0026#34;/opt/nri/plugins\u0026#34; # NRI plugin configuration directory to use. # nri_plugin_config_dir = \u0026#34;/etc/nri/conf.d\u0026#34; # Disable connections from externally launched NRI plugins. # nri_disable_connections = false # Timeout for a plugin to register itself with NRI. # nri_plugin_registration_timeout = \u0026#34;5s\u0026#34; # Timeout for a plugin to handle an NRI request. # nri_plugin_request_timeout = \u0026#34;2s\u0026#34; # Necessary information pertaining to container and pod stats reporting. [crio.stats] # The number of seconds between collecting pod and container stats. # If set to 0, the stats are collected on-demand instead. # stats_collection_period = 0 # The number of seconds between collecting pod/container stats and pod # sandbox metrics. If set to 0, the metrics/stats are collected on-demand instead. # collection_period = 0 mao@mao-cri-o:~$ cri-oの設定ファイルに追記する\n1 sudo nano /etc/crio/crio.conf 追記する1 1 2 3 [crio.runtime] conmon_cgroup = \u0026#34;pod\u0026#34; cgroup_manager = \u0026#34;cgroupfs\u0026#34; 追記する2 1 2 [crio.image] pause_image=\u0026#34;registry.k8s.io/pause:3.6\u0026#34; 1 2 3 [crio.image] - # pause_image = \u0026#34;registry.k8s.io/pause:3.9\u0026#34; + pause_image = \u0026#34;registry.k8s.io/pause:3.9\u0026#34; 1 2 - #default_runtime = \u0026#34;crun\u0026#34; + default_runtime = \u0026#34;runc\u0026#34; 設定を反映するためにリロードする\n1 sudo systemctl restart cri-o runCとCNIのインストール runC 1 2 sudo wget https://github.com/opencontainers/runc/releases/download/v1.1.13/runc.amd64 sudo install -m 755 runc.amd64 /usr/local/sbin/runc CNI 1 2 3 sudo wget https://github.com/containernetworking/plugins/releases/download/v1.5.1/cni-plugins-linux-amd64-v1.5.1.tgz sudo mkdir -p /opt/cni/bin sudo tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.5.1.tgz \u0026ldquo;kubelet\u0026rdquo;,\u0026ldquo;kubeadm\u0026rdquo;,\u0026ldquo;kubectl\u0026quot;のインストール リポジトリを追加する\n1 2 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list インストールをして、バージョンを固定する\n1 2 3 4 sudo apt update sudo apt install kubelet kubeadm kubectl sudo apt-mark hold kubelet kubeadm kubectl sudo apt-mark showhold Control-Planeでの作業1 クラスタの初期設定をする\n1 sudo kubeadm init --apiserver-advertise-address=192.168.10.55 --pod-network-cidr=10.128.0.0/16 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 mao@mao-cri-o:~$ sudo kubeadm init --apiserver-advertise-address=192.168.10.55 --pod-network-cidr=10.128.0.0/16 [init] Using Kubernetes version: v1.31.0 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action beforehand using \u0026#39;kubeadm config images pull\u0026#39; W0823 13:05:00.855747 12076 checks.go:846] detected that the sandbox image \u0026#34;registry.k8s.io/pause:3.6\u0026#34; of the container runtime is inconsistent with that used by kubeadm.It is recommended to use \u0026#34;registry.k8s.io/pause:3.10\u0026#34; as the CRI sandbox image. [certs] Using certificateDir folder \u0026#34;/etc/kubernetes/pki\u0026#34; [certs] Generating \u0026#34;ca\u0026#34; certificate and key [certs] Generating \u0026#34;apiserver\u0026#34; certificate and key [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local mao-cri-o] and IPs [10.96.0.1 192.168.10.55] [certs] Generating \u0026#34;apiserver-kubelet-client\u0026#34; certificate and key [certs] Generating \u0026#34;front-proxy-ca\u0026#34; certificate and key [certs] Generating \u0026#34;front-proxy-client\u0026#34; certificate and key [certs] Generating \u0026#34;etcd/ca\u0026#34; certificate and key [certs] Generating \u0026#34;etcd/server\u0026#34; certificate and key [certs] etcd/server serving cert is signed for DNS names [localhost mao-cri-o] and IPs [192.168.10.55 127.0.0.1 ::1] [certs] Generating \u0026#34;etcd/peer\u0026#34; certificate and key [certs] etcd/peer serving cert is signed for DNS names [localhost mao-cri-o] and IPs [192.168.10.55 127.0.0.1 ::1] [certs] Generating \u0026#34;etcd/healthcheck-client\u0026#34; certificate and key [certs] Generating \u0026#34;apiserver-etcd-client\u0026#34; certificate and key [certs] Generating \u0026#34;sa\u0026#34; key and public key [kubeconfig] Using kubeconfig folder \u0026#34;/etc/kubernetes\u0026#34; [kubeconfig] Writing \u0026#34;admin.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;super-admin.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;kubelet.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;controller-manager.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;scheduler.conf\u0026#34; kubeconfig file [etcd] Creating static Pod manifest for local etcd in \u0026#34;/etc/kubernetes/manifests\u0026#34; [control-plane] Using manifest folder \u0026#34;/etc/kubernetes/manifests\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-apiserver\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-controller-manager\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-scheduler\u0026#34; [kubelet-start] Writing kubelet environment file with flags to file \u0026#34;/var/lib/kubelet/kubeadm-flags.env\u0026#34; [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [kubelet-start] Starting the kubelet [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \u0026#34;/etc/kubernetes/manifests\u0026#34; [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 501.386574ms [api-check] Waiting for a healthy API server. This can take up to 4m0s [api-check] The API server is healthy after 5.501313388s [upload-config] Storing the configuration used in ConfigMap \u0026#34;kubeadm-config\u0026#34; in the \u0026#34;kube-system\u0026#34; Namespace [kubelet] Creating a ConfigMap \u0026#34;kubelet-config\u0026#34; in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node mao-cri-o as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node mao-cri-o as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] [bootstrap-token] Using token: chwroq.m0rdyxckr4zb0qpb [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the \u0026#34;cluster-info\u0026#34; ConfigMap in the \u0026#34;kube-public\u0026#34; namespace [kubelet-finalize] Updating \u0026#34;/etc/kubernetes/kubelet.conf\u0026#34; to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run \u0026#34;kubectl apply -f [podnetwork].yaml\u0026#34; with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.10.55:6443 --token chwroq.m0rdyxckr4zb0qpb \\ --discovery-token-ca-cert-hash sha256:e2abcd3797f4c417d228e4ffeb65f2215498933d2057d511701881b49625e629 mao@mao-cri-o:~$ joinコマンドの再表示 1 sudo kubeadm token create --print-join-command Control-Planeでの作業2 Calicoをインストールする\n1 2 wget https://raw.githubusercontent.com/projectcalico/calico/v3.28.1/manifests/tigera-operator.yaml kubectl create -f tigera-operator.yaml 1 wget https://raw.githubusercontent.com/projectcalico/calico/v3.28.1/manifests/custom-resources.yaml 1 2 - cidr: 192.168.0.0/16 + cidr: 10.128.0.0/16 1 kubectl apply -f custom-resources.yaml Woker-Nodeでの作業 クラスタにjoinするコマンドを実行する\n1 2 sudo kubeadm join 192.168.10.55:6443 --token chwroq.m0rdyxckr4zb0qpb \\ --discovery-token-ca-cert-hash sha256:e2abcd3797f4c417d228e4ffeb65f2215498933d2057d511701881b49625e629 確認 クラスタにjoinされていることを確認する\n1 kubectl get nodes -o wide 1 2 3 4 5 mao@mao-cri-o:~$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME mao-cri-o Ready control-plane 35m v1.31.0 192.168.10.55 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-41-generic cri-o://1.30.4 mao-cri-o-worker-node Ready \u0026lt;none\u0026gt; 32s v1.31.0 192.168.10.56 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-41-generic cri-o://1.30.4 mao@mao-cri-o:~$ コンテナランタイムが混ぜる containerdのクラスタにcri-oのNodeをjoinしようとしたらエラーになりコンテナランタイムを混ぜての構築はできなかった\n","date":"2024-08-25T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-cri-o/","title":"cri-oを使用してkubernetesを構築する"},{"content":"環境 Kubernetes 1.31.0 Helm v3.15.3 Traefik v3.1.2 HelmでTraefikのリポジトリを追加する リポジトリを追加してupdateする\n1 2 3 helm repo add traefik https://traefik.github.io/charts helm repo update helm repo list 1 2 3 4 5 6 7 8 9 10 11 12 mao@k8s-control-plane-01:~$ helm repo add traefik https://traefik.github.io/charts \u0026#34;traefik\u0026#34; has been added to your repositories mao@k8s-control-plane-01:~$ helm repo update Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the \u0026#34;kubernetes-dashboard\u0026#34; chart repository ...Successfully got an update from the \u0026#34;traefik\u0026#34; chart repository Update Complete. ⎈Happy Helming!⎈ mao@k8s-control-plane-01:~$ helm repo list NAME URL kubernetes-dashboard https://kubernetes.github.io/dashboard/ traefik https://traefik.github.io/charts mao@k8s-control-plane-01:~$ Ingress-Controllerをインストールする Ingress-Controllerを\u0026quot;ns-traefik\u0026quot;という\u0026quot;namespace\u0026quot;にインストールする\n作成されているか確認する\n1 2 helm install traefik traefik/traefik --create-namespace --namespace ns-traefik kubectl get services -A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 mao@k8s-control-plane-01:~$ helm install traefik traefik/traefik --create-namespace --namespace ns-traefik NAME: traefik LAST DEPLOYED: Tue Aug 20 11:51:28 2024 NAMESPACE: ns-traefik STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: traefik with docker.io/traefik:v3.1.2 has been deployed successfully on ns-traefik namespace ! mao@k8s-control-plane-01:~$ mao@k8s-control-plane-01:~$ kubectl get services -A NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE calico-apiserver calico-api ClusterIP 10.100.15.33 \u0026lt;none\u0026gt; 443/TCP 51d calico-system calico-kube-controllers-metrics ClusterIP None \u0026lt;none\u0026gt; 9094/TCP 51d calico-system calico-typha ClusterIP 10.111.65.20 \u0026lt;none\u0026gt; 5473/TCP 51d default kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 51d kube-system kube-dns ClusterIP 10.96.0.10 \u0026lt;none\u0026gt; 53/UDP,53/TCP,9153/TCP 51d metallb-system metallb-webhook-service ClusterIP 10.98.148.198 \u0026lt;none\u0026gt; 443/TCP 51d ns-traefik traefik LoadBalancer 10.98.137.158 192.168.10.55 80:31371/TCP,443:31450/TCP 2m32s mao@k8s-control-plane-01:~$ podが作成されていることを確認する\n1 kubectl get --namespace ns-traefik pod 1 2 3 4 mao@k8s-control-plane-01:~$ kubectl get --namespace ns-traefik pod NAME READY STATUS RESTARTS AGE traefik-6996c86bfd-kb284 1/1 Running 0 4m45s mao@k8s-control-plane-01:~$ Ingress,Serviceをデプロイする マニフェストファイルを作成する\ntraefik-ingress.yaml ingressとnginxのservice,podを一緒にデプロイする 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-traefik-test # ingressのnamespaceとserviceのnamespaceは同じにする #namespace: ns-traefik spec: rules: # hostを設定しないとIPアドレスでアクセスできる #- host: ingress1.internal - http: paths: - path: / pathType: Prefix backend: service: name: nginx-deployment-lb port: number: 83 - host: ingress1.internal http: paths: - path: / pathType: Prefix backend: service: name: nginx-deployment-lb port: number: 83 - host: ingress2.internal http: paths: - path: / pathType: Prefix backend: service: name: nginx-deployment-lb port: number: 83 --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment #namespace: ns-traefik labels: app: nginx spec: replicas: 10 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.27 ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: nginx-deployment-lb #namespace: ns-traefik spec: type: NodePort ports: - port: 83 targetPort: 80 selector: app: nginx デプロイする\n1 2 kubectl apply -f traefik-ingress.yaml kubectl get ingress -A アクセスできるか確認する IPアドレスとバーチャルホストでアクセスできるか確認する\n結果が帰ってくればOK\nアクセスできない場合はportがあっているかnamespaceが同じか等確認をする\n1 2 3 curl http://192.168.10.55 curl -H \u0026#39;Host:ingress1.internal\u0026#39; http://192.168.10.55 curl -H \u0026#39;Host:ingress2.internal\u0026#39; http://192.168.10.55 削除する 1 2 kubectl delete -f traefik-ingress.yaml helm uninstall traefik -n ns-traefik 1 2 kubectl get services -A helm ls -A 参考URL インストール（Helmを使用する） https://doc.traefik.io/traefik/getting-started/install-traefik/ Ingressのマニフェストファイル https://doc.traefik.io/traefik/providers/kubernetes-ingress/ https://doc.traefik.io/traefik/routing/providers/kubernetes-ingress/ Kubernetes で Traefik proxy を使う https://zenn.dev/zenogawa/articles/k8s_traefik_ingress 自宅の kubernetes に ingress-nginx を入れてみる https://konchangakita.hatenablog.com/entry/2020/07/13/220000 https://y-ohgi.com/introduction-kubernetes/3_objects/ingress/ https://qiita.com/dingtianhongjie/items/73980a3e9fbc8c7bc3cd ","date":"2024-08-21T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-ingress-traefik/","title":"KubernetesでTraefikを使用してIngressを使えるようにする"},{"content":"環境 Kubernetes 1.30.3 → 1.30.4 → 1.31.0 Control-Plane 3台 Woker-Node 2台 参考URL https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/ https://kubernetes.io/ja/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ 変更点の確認 https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.30.md#downloads-for-v1304 https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.31.md 問題なさそうなのでアップグレードします\n現状の構成 v1.30.3を使用している\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-control-plane-01 Ready control-plane 46d v1.30.3 192.168.10.41 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 k8s-control-plane-02 Ready control-plane 46d v1.30.3 192.168.10.44 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 k8s-control-plane-03 Ready control-plane 46d v1.30.3 192.168.10.46 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 k8s-worker-01 Ready \u0026lt;none\u0026gt; 46d v1.30.3 192.168.10.42 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 k8s-worker-02 Ready \u0026lt;none\u0026gt; 46d v1.30.3 192.168.10.43 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 mao@k8s-control-plane-01:~$ Control-Planeのアップグレード Control-Plane-01 アップグレードの際のコマンドを記載していきます\n1 2 3 4 kubectl drain --ignore-daemonsets k8s-control-plane-01 kubectl get nodes sudo apt update sudo apt-cache madison kubeadm 一旦v1.30.4にする 1 2 3 4 5 6 7 mao@k8s-control-plane-01:~$ sudo apt-cache madison kubeadm kubeadm | 1.30.4-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.3-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.2-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.1-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages mao@k8s-control-plane-01:~$ 1 2 3 4 5 6 7 8 9 10 11 12 sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.4-*\u0026#39; kubectl=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl kubeadm version sudo kubeadm upgrade plan sudo kubeadm upgrade apply v1.30.4 kubectl get nodes このままv1.31.0にアップグレードする 1 2 3 4 5 6 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list sudo apt update sudo apt-cache madison kubeadm 1 2 3 mao@k8s-control-plane-01:~$ sudo apt-cache madison kubeadm kubeadm | 1.31.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.31/deb Packages mao@k8s-control-plane-01:~$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.31.0-*\u0026#39; kubectl=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl kubeadm version sudo kubeadm upgrade plan sudo kubeadm upgrade apply v1.31.0 kubectl get nodes kubectl uncordon k8s-control-plane-01 kubectl get nodes Control-Plane-02 一旦v1.30.4にする 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 kubectl drain --ignore-daemonsets k8s-control-plane-02 kubectl get nodes sudo apt update sudo apt-cache madison kubeadm sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.4-*\u0026#39; kubectl=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl sudo kubeadm upgrade node kubectl get nodes このままv1.31.0にアップグレードする 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list sudo apt update sudo apt-cache madison kubeadm sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.31.0-*\u0026#39; kubectl=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl sudo kubeadm upgrade node kubectl get nodes kubectl uncordon k8s-control-plane-02 kubectl get nodes Control-Plane-03 一旦v1.30.4にする 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 kubectl drain --ignore-daemonsets k8s-control-plane-03 kubectl get nodes sudo apt update sudo apt-cache madison kubeadm sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.4-*\u0026#39; kubectl=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl sudo kubeadm upgrade node kubectl get nodes このままv1.31.0にアップグレードする 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list sudo apt update sudo apt-cache madison kubeadm sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.31.0-*\u0026#39; kubectl=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl sudo kubeadm upgrade node kubectl get nodes kubectl uncordon k8s-control-plane-03 kubectl get nodes Woker-Node Woker-Node-01 一旦v1.30.4にする Control-Planeでの作業 1 2 kubectl drain k8s-worker-01 --ignore-daemonsets kubectl get nodes Woker-Nodeでの作業 1 2 3 4 5 6 7 8 9 10 11 12 sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo kubeadm upgrade node sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.4-*\u0026#39; kubectl=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl sudo systemctl daemon-reload \u0026amp;\u0026amp; \\ sudo systemctl restart kubelet Control-Planeでの作業 1 kubectl get nodes このままv1.31.0にアップグレードする Woker-Nodeでの作業 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo kubeadm upgrade node sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.31.0-*\u0026#39; kubectl=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl sudo systemctl daemon-reload \u0026amp;\u0026amp; \\ sudo systemctl restart kubelet Control-Planeでの作業 1 2 3 kubectl get nodes kubectl uncordon k8s-worker-01 kubectl get nodes Woker-Node-02 一旦v1.30.4にする Control-Planeでの作業 1 2 kubectl drain k8s-worker-02 --ignore-daemonsets kubectl get nodes Woker-Nodeでの作業 1 2 3 4 5 6 7 8 9 10 11 12 sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo kubeadm upgrade node sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.4-*\u0026#39; kubectl=\u0026#39;1.30.4-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl sudo systemctl daemon-reload \u0026amp;\u0026amp; \\ sudo systemctl restart kubelet Control-Planeでの作業 1 kubectl get nodes このままv1.31.0にアップグレードする Woker-Nodeでの作業 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm sudo kubeadm upgrade node sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.31.0-*\u0026#39; kubectl=\u0026#39;1.31.0-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl sudo systemctl daemon-reload \u0026amp;\u0026amp; \\ sudo systemctl restart kubelet Control-Planeでの作業 1 2 3 kubectl get nodes kubectl uncordon k8s-worker-02 kubectl get nodes 最終確認 下記のコマンドで問題ないか確認する\n1 2 kubectl get nodes -o wide kubectl get pod -A -o wide 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 mao@k8s-control-plane-01:~$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-control-plane-01 Ready control-plane 46d v1.31.0 192.168.10.41 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 k8s-control-plane-02 Ready control-plane 46d v1.31.0 192.168.10.44 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 k8s-control-plane-03 Ready control-plane 46d v1.31.0 192.168.10.46 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 k8s-worker-01 Ready \u0026lt;none\u0026gt; 46d v1.31.0 192.168.10.42 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 k8s-worker-02 Ready \u0026lt;none\u0026gt; 46d v1.31.0 192.168.10.43 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-40-generic containerd://1.7.18 mao@k8s-control-plane-01:~$ kubectl get pod -A -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES calico-apiserver calico-apiserver-5f78767767-gfsdj 1/1 Running 0 2m44s 10.128.36.197 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-apiserver calico-apiserver-5f78767767-stslw 1/1 Running 0 9m44s 10.128.251.151 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-kube-controllers-5f5665469b-8rzw4 1/1 Running 0 2m44s 10.128.36.198 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-26sbk 1/1 Running 7 (26m ago) 46d 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-28xrv 1/1 Running 7 (55s ago) 46d 192.168.10.43 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-dc87d 1/1 Running 7 (47m ago) 46d 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-g2ks8 1/1 Running 8 (18m ago) 46d 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-l2ll2 1/1 Running 7 (4m46s ago) 46d 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-typha-5579b889c8-9r6v9 1/1 Running 0 4m30s 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-typha-5579b889c8-gd8sc 1/1 Running 0 17m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-typha-5579b889c8-n8nmp 1/1 Running 0 40s 192.168.10.43 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-2pzn6 2/2 Running 14 (4m46s ago) 46d 10.128.36.196 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-8b6ts 2/2 Running 14 (47m ago) 46d 10.128.251.150 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-9w86p 2/2 Running 14 (18m ago) 46d 10.128.204.146 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-cljz8 2/2 Running 14 (26m ago) 46d 10.128.194.204 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-jddlc 2/2 Running 14 (55s ago) 46d 10.128.118.120 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system coredns-7db6d8ff4d-7gv56 1/1 Running 0 9m44s 10.128.194.205 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system coredns-7db6d8ff4d-znvj7 1/1 Running 0 2m44s 10.128.36.202 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system etcd-k8s-control-plane-01 1/1 Running 0 46m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system etcd-k8s-control-plane-02 1/1 Running 0 25m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system etcd-k8s-control-plane-03 1/1 Running 0 17m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-apiserver-k8s-control-plane-01 1/1 Running 0 46m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-apiserver-k8s-control-plane-02 1/1 Running 1 (26m ago) 26m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-apiserver-k8s-control-plane-03 1/1 Running 1 (18m ago) 18m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-controller-manager-k8s-control-plane-01 1/1 Running 0 45m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-controller-manager-k8s-control-plane-02 1/1 Running 1 (26m ago) 26m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-controller-manager-k8s-control-plane-03 1/1 Running 1 (18m ago) 18m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-2h2wm 1/1 Running 1 (56s ago) 19m 192.168.10.43 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-746mc 1/1 Running 0 19m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-k2h9l 1/1 Running 1 (18m ago) 19m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-lvgb7 1/1 Running 1 (4m47s ago) 19m 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-xtm2m 1/1 Running 0 19m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-scheduler-k8s-control-plane-01 1/1 Running 0 45m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-scheduler-k8s-control-plane-02 1/1 Running 1 (26m ago) 26m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-scheduler-k8s-control-plane-03 1/1 Running 1 (18m ago) 18m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system controller-86f5578878-4whk5 1/1 Running 0 2m44s 10.128.36.200 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-2hccg 1/1 Running 13 (4m46s ago) 46d 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-cjz7j 1/1 Running 13 (26m ago) 46d 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-dpfpl 1/1 Running 13 (55s ago) 46d 192.168.10.43 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-gg452 1/1 Running 13 (18m ago) 46d 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-zj5x9 1/1 Running 13 (47m ago) 46d 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; tigera-operator tigera-operator-76ff79f7fd-8dgv4 1/1 Running 0 2m44s 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; mao@k8s-control-plane-01:~$ 全てNodeもpodも問題なく稼働しているのを確認したので、無事アップグレード完了\n","date":"2024-08-16T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-upgrade-v1-31-0/","title":"Kubernetesのクラスタをv1.30.3からv1.31.0へアップグレードをする"},{"content":" WindowsノートPCを売却するに当たり、データや開発環境をUbuntuへと移行しました\nその中でこのサイトの作成に使用しているHugoのインストール方法をメモしました\n環境 Ubuntu 24.04 LTS Intel Core i5-13500 (amd64) 参考URL https://gohugo.io/installation/linux/#prebuilt-binaries https://github.com/gohugoio/hugo/releases ダウンロード バイナリをダウンロードします\n1 wget https://github.com/gohugoio/hugo/releases/download/v0.132.1/hugo_extended_0.132.1_linux-amd64.tar.gz 展開・解凍 フォルダを作成して展開する\n1 2 mkdir hugo_extended_0.132.1_linux-amd64 tar -zxvf hugo_extended_0.132.1_linux-amd64.tar.gz -C ./hugo_extended_0.132.1_linux-amd64 パスの通っている場所へ移動し、バージョン確認 バイナリファイルを\u0026quot;/usr/local/bin\u0026quot;へ移動します\nその後バージョンの確認をします\n1 2 sudo mv hugo_extended_0.132.1_linux-amd64/hugo /usr/local/bin/hugo hugo version 1 2 3 mao@mao:~$ hugo version hugo v0.132.1-1bde700dfc0770bb11eb8445aff1ab5abdccb46e+extended linux/amd64 BuildDate=2024-08-13T10:10:10Z VendorInfo=gohugoio mao@mao:~$ アンインストール アンインストールする際は下記のコマンドを実行し、バイナリの場所を確認して削除します\n1 2 3 mao@mao:~$ which hugo /usr/local/bin/hugo mao@mao:~$ sudo rm /usr/local/bin/hugo ","date":"2024-08-15T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/ubuntu-install-hugo/","title":"UbuntuにHugoをインストールする"},{"content":"環境 Kubernetes 1.30.3 参考URL https://kubernetes.io/docs/concepts/storage/persistent-volumes/ https://kubernetes.io/docs/concepts/storage/storage-classes/ https://changineer.info/vmware/hypervisor/vmware_ubuntu_nfs.html https://blog.denet.co.jp/building-an-nfs-server-and-using-it-as-storage-from-kubernetes/ https://thinkit.co.jp/article/14195?page=0%2C1 NFSサーバーの構築 \u0026ldquo;nfs-server\u0026quot;を構築するサーバーで実行するコマンド\nインストールをする\n1 sudo apt install nfs-kernel-server NFSで公開するディレクトリの作成をする\n1 mkdir /home/mao/nfs 設定ファイルを編集する\n1 sudo nano /etc/exports 設定ファイルに下記を追加\n1 /home/mao/nfs 192.168.10.0/24(rw,sync,no_root_squash) 再起動と自動起動の有効化\n1 2 sudo systemctl restart nfs-server.service sudo systemctl enable nfs-server.service 設定が読み込まれているか確認する\n1 sudo exportfs nfs-commonをインストールする Kubernetesの全てのnode(Control-Plane,Woker-Node)に\u0026quot;nfs-common\u0026quot;をインストールする\n1 sudo apt install nfs-common 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 mao@k8s-control-plane-01:~$ sudo apt install nfs-common [sudo] password for mao: Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: keyutils libevent-core-2.1-7t64 libnfsidmap1 rpcbind Suggested packages: watchdog The following NEW packages will be installed: keyutils libevent-core-2.1-7t64 libnfsidmap1 nfs-common rpcbind 0 upgraded, 5 newly installed, 0 to remove and 30 not upgraded. Need to get 491 kB of archives. After this operation, 1680 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://jp.archive.ubuntu.com/ubuntu noble/main amd64 libevent-core-2.1-7t64 amd64 2.1.12-stable-9ubuntu2 [91.3 kB] Get:2 http://jp.archive.ubuntu.com/ubuntu noble/main amd64 libnfsidmap1 amd64 1:2.6.4-3ubuntu5 [48.2 kB] Get:3 http://jp.archive.ubuntu.com/ubuntu noble/main amd64 rpcbind amd64 1.2.6-7ubuntu2 [46.5 kB] Get:4 http://jp.archive.ubuntu.com/ubuntu noble/main amd64 keyutils amd64 1.6.3-3build1 [56.8 kB] Get:5 http://jp.archive.ubuntu.com/ubuntu noble/main amd64 nfs-common amd64 1:2.6.4-3ubuntu5 [248 kB] Fetched 491 kB in 2s (295 kB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package libevent-core-2.1-7t64:amd64. (Reading database ... 110850 files and directories currently installed.) Preparing to unpack .../libevent-core-2.1-7t64_2.1.12-stable-9ubuntu2_amd64.deb ... Unpacking libevent-core-2.1-7t64:amd64 (2.1.12-stable-9ubuntu2) ... Selecting previously unselected package libnfsidmap1:amd64. Preparing to unpack .../libnfsidmap1_1%3a2.6.4-3ubuntu5_amd64.deb ... Unpacking libnfsidmap1:amd64 (1:2.6.4-3ubuntu5) ... Selecting previously unselected package rpcbind. Preparing to unpack .../rpcbind_1.2.6-7ubuntu2_amd64.deb ... Unpacking rpcbind (1.2.6-7ubuntu2) ... Selecting previously unselected package keyutils. Preparing to unpack .../keyutils_1.6.3-3build1_amd64.deb ... Unpacking keyutils (1.6.3-3build1) ... Selecting previously unselected package nfs-common. Preparing to unpack .../nfs-common_1%3a2.6.4-3ubuntu5_amd64.deb ... Unpacking nfs-common (1:2.6.4-3ubuntu5) ... Setting up libnfsidmap1:amd64 (1:2.6.4-3ubuntu5) ... Setting up rpcbind (1.2.6-7ubuntu2) ... Created symlink /etc/systemd/system/multi-user.target.wants/rpcbind.service → /usr/lib/systemd/system/rpcbind.service. Created symlink /etc/systemd/system/sockets.target.wants/rpcbind.socket → /usr/lib/systemd/system/rpcbind.socket. Setting up keyutils (1.6.3-3build1) ... Setting up libevent-core-2.1-7t64:amd64 (2.1.12-stable-9ubuntu2) ... Setting up nfs-common (1:2.6.4-3ubuntu5) ... debconf: unable to initialize frontend: Dialog debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79.) debconf: falling back to frontend: Readline Creating config file /etc/idmapd.conf with new version debconf: unable to initialize frontend: Dialog debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79.) debconf: falling back to frontend: Readline debconf: unable to initialize frontend: Dialog debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79.) debconf: falling back to frontend: Readline Creating config file /etc/nfs.conf with new version info: Selecting UID from range 100 to 999 ... info: Adding system user `statd\u0026#39; (UID 106) ... info: Adding new user `statd\u0026#39; (UID 106) with group `nogroup\u0026#39; ... info: Not creating home directory `/var/lib/nfs\u0026#39;. Created symlink /etc/systemd/system/multi-user.target.wants/nfs-client.target → /usr/lib/systemd/system/nfs-client.target. Created symlink /etc/systemd/system/remote-fs.target.wants/nfs-client.target → /usr/lib/systemd/system/nfs-client.target. auth-rpcgss-module.service is a disabled or a static unit, not starting it. nfs-idmapd.service is a disabled or a static unit, not starting it. nfs-utils.service is a disabled or a static unit, not starting it. proc-fs-nfsd.mount is a disabled or a static unit, not starting it. rpc-gssd.service is a disabled or a static unit, not starting it. rpc-statd-notify.service is a disabled or a static unit, not starting it. rpc-statd.service is a disabled or a static unit, not starting it. rpc-svcgssd.service is a disabled or a static unit, not starting it. Processing triggers for libc-bin (2.39-0ubuntu8.2) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. mao@k8s-control-plane-01:~$ NFS-server上にPV(PersistentVolume)用のディレクトリを作成する nfs-serverで下記のコマンドを実行してPV用のディレクトリを作成する\n\u0026ldquo;pv0001\u0026rdquo; 1 mkdir /home/mao/nfs/pv0001 PV(PersistentVolume)のマニフェストファイルをデプロイする マニフェストファイルを作成する\ncreate-persistent-volume.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 apiVersion: v1 kind: PersistentVolume metadata: name: pv0001 annotations: volume.beta.kubernetes.io/storage-class: \u0026#34;slow\u0026#34; spec: capacity: storage: 5Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Recycle nfs: server: 192.168.10.17 path: /home/mao/nfs/pv0001 デプロイします\n1 kubectl apply -f create-persistent-volume.yaml 1 2 3 4 5 6 7 mao@k8s-control-plane-01:~$ kubectl apply -f create-persistent-volume.yaml Warning: metadata.annotations[volume.beta.kubernetes.io/storage-class]: deprecated since v1.8; use \u0026#34;storageClassName\u0026#34; attribute instead Warning: spec.persistentVolumeReclaimPolicy: The Recycle reclaim policy is deprecated. Instead, the recommended approach is to use dynamic provisioning. persistentvolume/pv0001 created mao@k8s-control-plane-01:~$ kubectl delete -f create-persistent-volume.yaml persistentvolume \u0026#34;pv0001\u0026#34; deleted mao@k8s-control-plane-01:~$ 警告が出たのでマニフェストファイルのストレージクラスとポリシーを修正します、合わせてアクセスモードも修正します\n\u0026ldquo;volume.beta.kubernetes.io/storage-class\u0026quot;を\u0026quot;spec.storageClassName\u0026quot;へ修正 \u0026ldquo;persistentVolumeReclaimPolicy\u0026quot;の\u0026quot;Recycle\u0026quot;を\u0026quot;Delete\u0026quot;へ修正 \u0026ldquo;spec.accessModes\u0026quot;を\u0026quot;ReadWriteOnce\u0026quot;を\u0026quot;ReadWriteMany\u0026quot;へ修正 \u0026ldquo;spec.nfs.server\u0026quot;は\u0026quot;nfs-server\u0026quot;のIPアドレスを指定する \u0026ldquo;spec.nfs.path\u0026quot;はPV用ディレクトリを指定します create-persistent-volume.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 apiVersion: v1 kind: PersistentVolume metadata: name: pv0001 spec: capacity: storage: 5Gi volumeMode: Filesystem accessModes: #- ReadWriteOnce - ReadWriteMany persistentVolumeReclaimPolicy: Delete storageClassName: slow nfs: server: 192.168.10.17 path: /home/mao/nfs/pv0001 デプロイします\n1 2 3 mao@k8s-control-plane-01:~$ kubectl apply -f create-persistent-volume.yaml persistentvolume/pv0001 created mao@k8s-control-plane-01:~$ 作成されているか確認をします\n1 kubectl get pv 1 2 3 4 mao@k8s-control-plane-01:~$ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE pv0001 5Gi RWX Delete Available slow \u0026lt;unset\u0026gt; 2m5s mao@k8s-control-plane-01:~$ \u0026ldquo;STATUS\u0026quot;が\u0026quot;Available\u0026quot;になっていればOK 無事作成されました\nPVC(PersistentVolumeClaim)をデプロイする マニフェストファイルを作成します\ncreate-persistent-volume-claim.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 apiVersion: v1 kind: PersistentVolumeClaim metadata: name: myclaim spec: accessModes: #- ReadWriteOnce - ReadWriteMany volumeMode: Filesystem resources: requests: storage: 5Gi storageClassName: slow デプロイします\n1 kubectl apply -f create-persistent-volume-claim.yaml 1 2 3 mao@k8s-control-plane-01:~$ kubectl apply -f create-persistent-volume-claim.yaml persistentvolumeclaim/myclaim created mao@k8s-control-plane-01:~$ 作成されているか確認します\n1 kubectl get pvc 1 2 3 4 mao@k8s-control-plane-01:~$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE myclaim Bound pv0001 5Gi RWX slow \u0026lt;unset\u0026gt; 31s mao@k8s-control-plane-01:~$ \u0026ldquo;STATUS\u0026quot;が\u0026quot;Bound\u0026quot;になっていればOK 無事作成されました\nこれでPV,PVCが必要なアプリケーションをデプロイできるようになりました\nPVC、PVの削除 削除していきます 1 2 kubectl delete -f create-persistent-volume-claim.yaml kubectl delete -f create-persistent-volume.yaml 1 2 3 4 5 6 7 8 9 mao@k8s-control-plane-01:~$ kubectl delete -f create-persistent-volume-claim.yaml persistentvolumeclaim \u0026#34;myclaim\u0026#34; deleted mao@k8s-control-plane-01:~$ kubectl delete -f create-persistent-volume.yaml persistentvolume \u0026#34;pv0001\u0026#34; deleted mao@k8s-control-plane-01:~$ kubectl get pvc No resources found in default namespace. mao@k8s-control-plane-01:~$ kubectl get pv No resources found mao@k8s-control-plane-01:~$ 問題なく削除されたことを確認\n完了 ","date":"2024-08-14T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-pv-pvc/","title":"NFS-serverを構築してPV(PersistentVolume)を作成する"},{"content":"環境 Kubernetes 1.30.2 → 1.30.3 Control-Plane 3台 Woker-Node 2台 現状の構成 v1.30.2を使用している\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-control-plane-01 Ready control-plane 42d v1.30.2 192.168.10.41 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-36-generic containerd://1.7.18 k8s-control-plane-02 Ready control-plane 42d v1.30.2 192.168.10.44 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-36-generic containerd://1.7.18 k8s-control-plane-03 Ready control-plane 42d v1.30.2 192.168.10.46 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-39-generic containerd://1.7.18 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 192.168.10.42 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-39-generic containerd://1.7.18 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 192.168.10.43 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-39-generic containerd://1.7.18 mao@k8s-control-plane-01:~$ 手順 公式サイトの手順でアップグレードしていく\nhttps://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/ 以下の順番でアップグレードしていく\nControl-Planeをアップグレード Control-Plane-01をアップグレード Control-Plane-02をアップグレード Control-Plane-03をアップグレード Woker-Nodeをアップグレード Woker-Node-01をアップグレード Woker-Node-02をアップグレード Control-Planeのアップグレード node上あるpodを別にnodeへ移動していく node名を確認する\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.2 k8s-control-plane-02 Ready control-plane 42d v1.30.2 k8s-control-plane-03 Ready control-plane 42d v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-01:~$ 最初に\u0026quot;k8s-control-plane-01\u0026quot;をアップグレードする\n1 2 kubectl drain --ignore-daemonsets \u0026lt;node name\u0026gt; kubectl drain --ignore-daemonsets k8s-control-plane-01 1 2 3 4 5 6 7 8 9 10 11 mao@k8s-control-plane-01:~$ kubectl drain --ignore-daemonsets k8s-control-plane-01 node/k8s-control-plane-01 cordoned Warning: ignoring DaemonSet-managed Pods: calico-system/calico-node-dc87d, calico-system/csi-node-driver-8b6ts, kube-system/kube-proxy-9ng8c, metallb-system/speaker-zj5x9 evicting pod kube-system/coredns-7db6d8ff4d-w9f4s evicting pod calico-apiserver/calico-apiserver-5f78767767-hl7s7 evicting pod kube-system/coredns-7db6d8ff4d-vdlkw pod/calico-apiserver-5f78767767-hl7s7 evicted pod/coredns-7db6d8ff4d-vdlkw evicted pod/coredns-7db6d8ff4d-w9f4s evicted node/k8s-control-plane-01 drained mao@k8s-control-plane-01:~$ \u0026ldquo;k8s-control-plane-01\u0026quot;の\u0026quot;STATUS\u0026quot;が\u0026quot;SchedulingDisabled\u0026quot;になっているか確認する\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready,SchedulingDisabled control-plane 42d v1.30.2 k8s-control-plane-02 Ready control-plane 42d v1.30.2 k8s-control-plane-03 Ready control-plane 42d v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-01:~$ kubeadm kubelet kubectl をアップデートする アップデートする対象のバージョンがあるか確認する\n1 2 sudo apt update sudo apt-cache madison kubeadm \u0026ldquo;1.30.3\u0026quot;があることを確認\n1 2 3 4 5 6 mao@k8s-control-plane-01:~$ sudo apt-cache madison kubeadm kubeadm | 1.30.3-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.2-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.1-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages mao@k8s-control-plane-01:~$ \u0026ldquo;kubeadm\u0026quot;を\u0026quot;1.30.3\u0026quot;にアップデートする\n1 2 3 sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 mao@k8s-control-plane-01:~$ sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm Canceled hold on kubeadm. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:4 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubeadm\u0026#39; The following packages will be upgraded: kubeadm 1 upgraded, 0 newly installed, 0 to remove and 32 not upgraded. Need to get 10.4 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubeadm 1.30.3-1.1 [10.4 MB] Fetched 10.4 MB in 0s (31.6 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110850 files and directories currently installed.) Preparing to unpack .../kubeadm_1.30.3-1.1_amd64.deb ... Unpacking kubeadm (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubeadm (1.30.3-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-36-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... Service restarts being deferred: systemctl restart systemd-logind.service systemctl restart unattended-upgrades.service No containers need to be restarted. User sessions running outdated binaries: mao @ session #1: sshd[4320] mao @ user manager service: systemd[4324] No VM guests are running outdated hypervisor (qemu) binaries on this host. kubeadm set on hold. mao@k8s-control-plane-01:~$ \u0026ldquo;kubelet\u0026quot;と\u0026quot;kubectl\u0026quot;もアップデートする\n1 2 3 sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.3-*\u0026#39; kubectl=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 mao@k8s-control-plane-01:~$ sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.3-*\u0026#39; kubectl=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl kubelet was already not on hold. kubectl was already not on hold. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:3 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:4 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubelet\u0026#39; Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubectl\u0026#39; The following packages will be upgraded: kubectl kubelet 2 upgraded, 0 newly installed, 0 to remove and 30 not upgraded. Need to get 28.9 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubectl 1.30.3-1.1 [10.8 MB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubelet 1.30.3-1.1 [18.1 MB] Fetched 28.9 MB in 1s (57.2 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110850 files and directories currently installed.) Preparing to unpack .../kubectl_1.30.3-1.1_amd64.deb ... Unpacking kubectl (1.30.3-1.1) over (1.30.2-1.1) ... Preparing to unpack .../kubelet_1.30.3-1.1_amd64.deb ... Unpacking kubelet (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubectl (1.30.3-1.1) ... Setting up kubelet (1.30.3-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-36-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... systemctl restart kubelet.service Service restarts being deferred: systemctl restart systemd-logind.service systemctl restart unattended-upgrades.service No containers need to be restarted. User sessions running outdated binaries: mao @ session #1: sshd[4320] mao @ user manager service: systemd[4324] No VM guests are running outdated hypervisor (qemu) binaries on this host. kubelet set on hold. kubectl set on hold. mao@k8s-control-plane-01:~$ \u0026ldquo;kubeadm\u0026quot;のバージョンを確認する\n1 kubeadm version 1 2 3 mao@k8s-control-plane-01:~$ kubeadm version kubeadm version: \u0026amp;version.Info{Major:\u0026#34;1\u0026#34;, Minor:\u0026#34;30\u0026#34;, GitVersion:\u0026#34;v1.30.3\u0026#34;, GitCommit:\u0026#34;6fc0a69044f1ac4c13841ec4391224a2df241460\u0026#34;, GitTreeState:\u0026#34;clean\u0026#34;, BuildDate:\u0026#34;2024-07-16T23:53:15Z\u0026#34;, GoVersion:\u0026#34;go1.22.5\u0026#34;, Compiler:\u0026#34;gc\u0026#34;, Platform:\u0026#34;linux/amd64\u0026#34;} mao@k8s-control-plane-01:~$ \u0026ldquo;v1.30.3\u0026quot;となっていることを確認\nアップグレードプランを確認する\n1 sudo kubeadm upgrade plan 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 mao@k8s-control-plane-01:~$ sudo kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [upgrade] Running cluster health checks [upgrade] Fetching available versions to upgrade to [upgrade/versions] Cluster version: 1.30.2 [upgrade/versions] kubeadm version: v1.30.3 [upgrade/versions] Target version: v1.30.3 [upgrade/versions] Latest version in the v1.30 series: v1.30.3 Components that must be upgraded manually after you have upgraded the control plane with \u0026#39;kubeadm upgrade apply\u0026#39;: COMPONENT NODE CURRENT TARGET kubelet k8s-control-plane-02 v1.30.2 v1.30.3 kubelet k8s-control-plane-03 v1.30.2 v1.30.3 kubelet k8s-worker-01 v1.30.2 v1.30.3 kubelet k8s-worker-02 v1.30.2 v1.30.3 kubelet k8s-control-plane-01 v1.30.3 v1.30.3 Upgrade to the latest version in the v1.30 series: COMPONENT NODE CURRENT TARGET kube-apiserver k8s-control-plane-01 v1.30.2 v1.30.3 kube-apiserver k8s-control-plane-02 v1.30.2 v1.30.3 kube-apiserver k8s-control-plane-03 v1.30.2 v1.30.3 kube-controller-manager k8s-control-plane-01 v1.30.2 v1.30.3 kube-controller-manager k8s-control-plane-02 v1.30.2 v1.30.3 kube-controller-manager k8s-control-plane-03 v1.30.2 v1.30.3 kube-scheduler k8s-control-plane-01 v1.30.2 v1.30.3 kube-scheduler k8s-control-plane-02 v1.30.2 v1.30.3 kube-scheduler k8s-control-plane-03 v1.30.2 v1.30.3 kube-proxy 1.30.2 v1.30.3 CoreDNS v1.11.1 v1.11.1 etcd k8s-control-plane-01 3.5.12-0 3.5.12-0 etcd k8s-control-plane-02 3.5.12-0 3.5.12-0 etcd k8s-control-plane-03 3.5.12-0 3.5.12-0 You can now apply the upgrade by executing the following command: kubeadm upgrade apply v1.30.3 _____________________________________________________________________ The table below shows the current state of component configs as understood by this version of kubeadm. Configs that have a \u0026#34;yes\u0026#34; mark in the \u0026#34;MANUAL UPGRADE REQUIRED\u0026#34; column require manual config upgrade or resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually upgrade to is denoted in the \u0026#34;PREFERRED VERSION\u0026#34; column. API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED kubeproxy.config.k8s.io v1alpha1 v1alpha1 no kubelet.config.k8s.io v1beta1 v1beta1 no _____________________________________________________________________ mao@k8s-control-plane-01:~$ 実際にアップグレードする\n1 sudo kubeadm upgrade apply v1.30.3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 mao@k8s-control-plane-01:~$ sudo kubeadm upgrade apply v1.30.3 [preflight] Running pre-flight checks. [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [upgrade] Running cluster health checks [upgrade/version] You have chosen to change the cluster version to \u0026#34;v1.30.3\u0026#34; [upgrade/versions] Cluster version: v1.30.2 [upgrade/versions] kubeadm version: v1.30.3 [upgrade] Are you sure you want to proceed? [y/N]: y [upgrade/prepull] Pulling images required for setting up a Kubernetes cluster [upgrade/prepull] This might take a minute or two, depending on the speed of your internet connection [upgrade/prepull] You can also perform this action in beforehand using \u0026#39;kubeadm config images pull\u0026#39; [upgrade/apply] Upgrading your Static Pod-hosted control plane to version \u0026#34;v1.30.3\u0026#34; (timeout: 5m0s)... [upgrade/etcd] Upgrading to TLS for etcd [upgrade/staticpods] Preparing for \u0026#34;etcd\u0026#34; upgrade [upgrade/staticpods] Current and new manifests of etcd are equal, skipping upgrade [upgrade/etcd] Waiting for etcd to become available [upgrade/staticpods] Writing new Static Pod manifests to \u0026#34;/etc/kubernetes/tmp/kubeadm-upgraded-manifests890193377\u0026#34; [upgrade/staticpods] Preparing for \u0026#34;kube-apiserver\u0026#34; upgrade [upgrade/staticpods] Renewing apiserver certificate [upgrade/staticpods] Renewing apiserver-kubelet-client certificate [upgrade/staticpods] Renewing front-proxy-client certificate [upgrade/staticpods] Renewing apiserver-etcd-client certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-apiserver.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-05-10/kube-apiserver.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-apiserver [upgrade/staticpods] Component \u0026#34;kube-apiserver\u0026#34; upgraded successfully! [upgrade/staticpods] Preparing for \u0026#34;kube-controller-manager\u0026#34; upgrade [upgrade/staticpods] Renewing controller-manager.conf certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-controller-manager.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-05-10/kube-controller-manager.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-controller-manager [upgrade/staticpods] Component \u0026#34;kube-controller-manager\u0026#34; upgraded successfully! [upgrade/staticpods] Preparing for \u0026#34;kube-scheduler\u0026#34; upgrade [upgrade/staticpods] Renewing scheduler.conf certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-scheduler.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-05-10/kube-scheduler.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-scheduler [upgrade/staticpods] Component \u0026#34;kube-scheduler\u0026#34; upgraded successfully! [upload-config] Storing the configuration used in ConfigMap \u0026#34;kubeadm-config\u0026#34; in the \u0026#34;kube-system\u0026#34; Namespace [kubelet] Creating a ConfigMap \u0026#34;kubelet-config\u0026#34; in namespace kube-system with the configuration for the kubelets in the cluster [upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config745490494/config.yaml [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [upgrade/addons] skip upgrade addons because control plane instances [k8s-control-plane-02 k8s-control-plane-03] have not been upgraded [upgrade/successful] SUCCESS! Your cluster was upgraded to \u0026#34;v1.30.3\u0026#34;. Enjoy! [upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven\u0026#39;t already done so. mao@k8s-control-plane-01:~$ 無事アップグレードされました\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready,SchedulingDisabled control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.2 k8s-control-plane-03 Ready control-plane 42d v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-01:~$ nodeからpodを退避させているのを解除する \u0026ldquo;kubectl drain\u0026quot;を解除する\n1 2 kubectl uncordon \u0026lt;node name\u0026gt; kubectl uncordon k8s-control-plane-01 1 2 3 mao@k8s-control-plane-01:~$ kubectl uncordon k8s-control-plane-01 node/k8s-control-plane-01 uncordoned mao@k8s-control-plane-01:~$ \u0026ldquo;STATUS\u0026quot;に\u0026quot;SchedulingDisabled\u0026quot;が表示されなくなったことを確認\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.2 k8s-control-plane-03 Ready control-plane 42d v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-01:~$ 他のContol-Planeもアップグレードをする k8s-control-plane-02 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 mao@k8s-control-plane-02:~$ kubectl drain --ignore-daemonsets k8s-control-plane-0 2 node/k8s-control-plane-02 cordoned Warning: ignoring DaemonSet-managed Pods: calico-system/calico-node-26sbk, calico-system/csi-node-driver-cljz8, kube-system/kube-proxy-xkvj7, metallb-system/speaker-cjz7j evicting pod calico-system/calico-typha-5579b889c8-kbqk7 evicting pod calico-apiserver/calico-apiserver-5f78767767-89z5t pod/calico-apiserver-5f78767767-89z5t evicted pod/calico-typha-5579b889c8-kbqk7 evicted node/k8s-control-plane-02 drained mao@k8s-control-plane-02:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready,SchedulingDisabled control-plane 42d v1.30.2 k8s-control-plane-03 Ready control-plane 42d v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-02:~$ sudo apt update [sudo] password for mao: Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://security.ubuntu.com/ubuntu noble-security InRelease Get:4 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB] Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Get:6 http://jp.archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [344 kB] Get:7 http://jp.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [321 kB] Fetched 791 kB in 3s (298 kB/s) Reading package lists... Done Building dependency tree... Done Reading state information... Done 31 packages can be upgraded. Run \u0026#39;apt list --upgradable\u0026#39; to see them. mao@k8s-control-plane-02:~$ sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm Canceled hold on kubeadm. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:4 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubeadm\u0026#39; The following packages will be upgraded: kubeadm 1 upgraded, 0 newly installed, 0 to remove and 30 not upgraded. Need to get 10.4 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubeadm 1.30.3-1.1 [10.4 MB] Fetched 10.4 MB in 0s (30.5 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110849 files and directories currently installed.) Preparing to unpack .../kubeadm_1.30.3-1.1_amd64.deb ... Unpacking kubeadm (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubeadm (1.30.3-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-36-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... Service restarts being deferred: systemctl restart systemd-logind.service systemctl restart unattended-upgrades.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. kubeadm set on hold. mao@k8s-control-plane-02:~$ sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.3-*\u0026#39; kubectl=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl Canceled hold on kubelet. Canceled hold on kubectl. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubelet\u0026#39; Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubectl\u0026#39; The following packages will be upgraded: kubectl kubelet 2 upgraded, 0 newly installed, 0 to remove and 28 not upgraded. Need to get 28.9 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubectl 1.30.3-1.1 [10.8 MB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubelet 1.30.3-1.1 [18.1 MB] Fetched 28.9 MB in 1s (55.7 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110849 files and directories currently installed.) Preparing to unpack .../kubectl_1.30.3-1.1_amd64.deb ... Unpacking kubectl (1.30.3-1.1) over (1.30.2-1.1) ... Preparing to unpack .../kubelet_1.30.3-1.1_amd64.deb ... Unpacking kubelet (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubectl (1.30.3-1.1) ... Setting up kubelet (1.30.3-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-36-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... systemctl restart kubelet.service Service restarts being deferred: systemctl restart systemd-logind.service systemctl restart unattended-upgrades.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. kubelet set on hold. kubectl set on hold. mao@k8s-control-plane-02:~$ kubeadm version kubeadm version: \u0026amp;version.Info{Major:\u0026#34;1\u0026#34;, Minor:\u0026#34;30\u0026#34;, GitVersion:\u0026#34;v1.30.3\u0026#34;, GitCommit:\u0026#34;6fc0a69044f1ac4c13841ec4391224a2df241460\u0026#34;, GitTreeState:\u0026#34;clean\u0026#34;, BuildDate:\u0026#34;2024-07-16T23:53:15Z\u0026#34;, GoVersion:\u0026#34;go1.22.5\u0026#34;, Compiler:\u0026#34;gc\u0026#34;, Platform:\u0026#34;linux/amd64\u0026#34;} mao@k8s-control-plane-02:~$ sudo kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [upgrade] Running cluster health checks [upgrade] Fetching available versions to upgrade to W0811 23:15:18.916628 16854 compute.go:93] Different API server versions in the cluster were discovered: v1.30.3 on nodes [k8s-control-plane-01], v1.30.2 on nodes [k8s-control-plane-02 k8s-control-plane-03]. Please upgrade your control plane nodes to the same version of Kubernetes [upgrade/versions] Cluster version: 1.30.3 [upgrade/versions] kubeadm version: v1.30.3 [upgrade/versions] Target version: v1.30.3 [upgrade/versions] Latest version in the v1.30 series: v1.30.3 mao@k8s-control-plane-02:~$ sudo kubeadm upgrade apply v1.30.3 [preflight] Running pre-flight checks. [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [upgrade] Running cluster health checks [upgrade/version] You have chosen to change the cluster version to \u0026#34;v1.30.3\u0026#34; [upgrade/versions] Cluster version: v1.30.2 [upgrade/versions] kubeadm version: v1.30.3 [upgrade] Are you sure you want to proceed? [y/N]: y [upgrade/prepull] Pulling images required for setting up a Kubernetes cluster [upgrade/prepull] This might take a minute or two, depending on the speed of your internet connection [upgrade/prepull] You can also perform this action in beforehand using \u0026#39;kubeadm config images pull\u0026#39; [upgrade/apply] Upgrading your Static Pod-hosted control plane to version \u0026#34;v1.30.3\u0026#34; (timeout: 5m0s)... [upgrade/etcd] Upgrading to TLS for etcd [upgrade/staticpods] Preparing for \u0026#34;etcd\u0026#34; upgrade [upgrade/staticpods] Renewing etcd-server certificate [upgrade/staticpods] Renewing etcd-peer certificate [upgrade/staticpods] Renewing etcd-healthcheck-client certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/etcd.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-15-44/etcd.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=etcd [upgrade/staticpods] Component \u0026#34;etcd\u0026#34; upgraded successfully! [upgrade/etcd] Waiting for etcd to become available [upgrade/staticpods] Writing new Static Pod manifests to \u0026#34;/etc/kubernetes/tmp/kubeadm-upgraded-manifests3449985093\u0026#34; [upgrade/staticpods] Preparing for \u0026#34;kube-apiserver\u0026#34; upgrade [upgrade/staticpods] Renewing apiserver certificate [upgrade/staticpods] Renewing apiserver-kubelet-client certificate [upgrade/staticpods] Renewing front-proxy-client certificate [upgrade/staticpods] Renewing apiserver-etcd-client certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-apiserver.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-15-44/kube-apiserver.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-apiserver [upgrade/staticpods] Component \u0026#34;kube-apiserver\u0026#34; upgraded successfully! [upgrade/staticpods] Preparing for \u0026#34;kube-controller-manager\u0026#34; upgrade [upgrade/staticpods] Renewing controller-manager.conf certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-controller-manager.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-15-44/kube-controller-manager.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-controller-manager [upgrade/staticpods] Component \u0026#34;kube-controller-manager\u0026#34; upgraded successfully! [upgrade/staticpods] Preparing for \u0026#34;kube-scheduler\u0026#34; upgrade [upgrade/staticpods] Renewing scheduler.conf certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-scheduler.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-15-44/kube-scheduler.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-scheduler [upgrade/staticpods] Component \u0026#34;kube-scheduler\u0026#34; upgraded successfully! [upload-config] Storing the configuration used in ConfigMap \u0026#34;kubeadm-config\u0026#34; in the \u0026#34;kube-system\u0026#34; Namespace [kubelet] Creating a ConfigMap \u0026#34;kubelet-config\u0026#34; in namespace kube-system with the configuration for the kubelets in the cluster [upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config534750710/config.yaml [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [upgrade/addons] skip upgrade addons because control plane instances [k8s-control-plane-03] have not been upgraded [upgrade/successful] SUCCESS! Your cluster was upgraded to \u0026#34;v1.30.3\u0026#34;. Enjoy! [upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven\u0026#39;t already done so. mao@k8s-control-plane-02:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready,SchedulingDisabled control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-02:~$ kubectl uncordon k8s-control-plane-02 node/k8s-control-plane-02 uncordoned mao@k8s-control-plane-02:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-02:~$ k8s-control-plane-03 \u0026ldquo;sudo kubeadm upgrade plan\u0026quot;と\u0026quot;sudo kubeadm upgrade apply v1.30.3\u0026quot;の変わりに\u0026quot;sudo kubeadm upgrade node\u0026quot;を実行する 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 mao@k8s-control-plane-03:~$ kubectl drain --ignore-daemonsets k8s-control-plane-03 node/k8s-control-plane-03 cordoned Warning: ignoring DaemonSet-managed Pods: calico-system/calico-node-g2ks8, calico-system/csi-node-driver-9w86p, kube-system/kube-proxy-4l8w6, metallb-system/speaker-gg452 evicting pod tigera-operator/tigera-operator-76ff79f7fd-tc9t7 evicting pod calico-system/calico-kube-controllers-5f5665469b-nh6qm pod/tigera-operator-76ff79f7fd-tc9t7 evicted pod/calico-kube-controllers-5f5665469b-nh6qm evicted node/k8s-control-plane-03 drained mao@k8s-control-plane-03:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready,SchedulingDisabled control-plane 42d v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-03:~$ sudo apt update [sudo] password for mao: Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Get:3 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB] Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Get:6 http://jp.archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [344 kB] Get:7 http://jp.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [321 kB] Fetched 791 kB in 2s (466 kB/s) Reading package lists... Done Building dependency tree... Done Reading state information... Done 33 packages can be upgraded. Run \u0026#39;apt list --upgradable\u0026#39; to see them. mao@k8s-control-plane-03:~$ sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm Canceled hold on kubeadm. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:3 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:4 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubeadm\u0026#39; The following packages will be upgraded: kubeadm 1 upgraded, 0 newly installed, 0 to remove and 32 not upgraded. Need to get 10.4 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubeadm 1.30.3-1.1 [10.4 MB] Fetched 10.4 MB in 0s (31.7 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110850 files and directories currently installed.) Preparing to unpack .../kubeadm_1.30.3-1.1_amd64.deb ... Unpacking kubeadm (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubeadm (1.30.3-1.1) ... Scanning processes... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-39-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. kubeadm set on hold. mao@k8s-control-plane-03:~$ sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.3-*\u0026#39; kubectl=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl Canceled hold on kubelet. Canceled hold on kubectl. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:4 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubelet\u0026#39; Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubectl\u0026#39; The following packages will be upgraded: kubectl kubelet 2 upgraded, 0 newly installed, 0 to remove and 30 not upgraded. Need to get 28.9 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubectl 1.30.3-1.1 [10.8 MB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubelet 1.30.3-1.1 [18.1 MB] Fetched 28.9 MB in 0s (58.1 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110850 files and directories currently installed.) Preparing to unpack .../kubectl_1.30.3-1.1_amd64.deb ... Unpacking kubectl (1.30.3-1.1) over (1.30.2-1.1) ... Preparing to unpack .../kubelet_1.30.3-1.1_amd64.deb ... Unpacking kubelet (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubectl (1.30.3-1.1) ... Setting up kubelet (1.30.3-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-39-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... systemctl restart kubelet.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. kubelet set on hold. kubectl set on hold. mao@k8s-control-plane-03:~$ kubeadm version kubeadm version: \u0026amp;version.Info{Major:\u0026#34;1\u0026#34;, Minor:\u0026#34;30\u0026#34;, GitVersion:\u0026#34;v1.30.3\u0026#34;, GitCommit:\u0026#34;6fc0a69044f1ac4c13841ec4391224a2df241460\u0026#34;, GitTreeState:\u0026#34;clean\u0026#34;, BuildDate:\u0026#34;2024-07-16T23:53:15Z\u0026#34;, GoVersion:\u0026#34;go1.22.5\u0026#34;, Compiler:\u0026#34;gc\u0026#34;, Platform:\u0026#34;linux/amd64\u0026#34;} mao@k8s-control-plane-03:~$ sudo kubeadm upgrade node [upgrade] Reading configuration from the cluster... [upgrade] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using \u0026#39;kubeadm config images pull\u0026#39; [upgrade] Upgrading your Static Pod-hosted control plane instance to version \u0026#34;v1.30.3\u0026#34;... [upgrade/etcd] Upgrading to TLS for etcd [upgrade/staticpods] Preparing for \u0026#34;etcd\u0026#34; upgrade [upgrade/staticpods] Renewing etcd-server certificate [upgrade/staticpods] Renewing etcd-peer certificate [upgrade/staticpods] Renewing etcd-healthcheck-client certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/etcd.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-21-58/etcd.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=etcd [upgrade/staticpods] Component \u0026#34;etcd\u0026#34; upgraded successfully! [upgrade/etcd] Waiting for etcd to become available [upgrade/staticpods] Writing new Static Pod manifests to \u0026#34;/etc/kubernetes/tmp/kubeadm-upgraded-manifests3409919858\u0026#34; [upgrade/staticpods] Preparing for \u0026#34;kube-apiserver\u0026#34; upgrade [upgrade/staticpods] Renewing apiserver certificate [upgrade/staticpods] Renewing apiserver-kubelet-client certificate [upgrade/staticpods] Renewing front-proxy-client certificate [upgrade/staticpods] Renewing apiserver-etcd-client certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-apiserver.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-21-58/kube-apiserver.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-apiserver [upgrade/staticpods] Component \u0026#34;kube-apiserver\u0026#34; upgraded successfully! [upgrade/staticpods] Preparing for \u0026#34;kube-controller-manager\u0026#34; upgrade [upgrade/staticpods] Renewing controller-manager.conf certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-controller-manager.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-21-58/kube-controller-manager.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-controller-manager [upgrade/staticpods] Component \u0026#34;kube-controller-manager\u0026#34; upgraded successfully! [upgrade/staticpods] Preparing for \u0026#34;kube-scheduler\u0026#34; upgrade [upgrade/staticpods] Renewing scheduler.conf certificate [upgrade/staticpods] Moved new manifest to \u0026#34;/etc/kubernetes/manifests/kube-scheduler.yaml\u0026#34; and backed up old manifest to \u0026#34;/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-11-23-21-58/kube-scheduler.yaml\u0026#34; [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 3 Pods for label selector component=kube-scheduler [upgrade/staticpods] Component \u0026#34;kube-scheduler\u0026#34; upgraded successfully! [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy [upgrade] The control plane instance for this node was successfully updated! [upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config1587591676/config.yaml [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [upgrade] The configuration for this node was successfully updated! [upgrade] Now you should go ahead and upgrade the kubelet package using your package manager. mao@k8s-control-plane-03:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready,SchedulingDisabled control-plane 42d v1.30.3 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-03:~$ kubectl uncordon k8s-control-plane-03 node/k8s-control-plane-03 uncordoned mao@k8s-control-plane-03:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.3 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-03:~$ これでControl-Planeのアップグレードは完了 Worker-Nodeをアップグレードする kubeadmをアップデート \u0026ldquo;kubeadm\u0026quot;をアップデートする\n1 2 3 sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 mao@k8s-worker-01:~$ sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm [sudo] password for mao: Canceled hold on kubeadm. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:3 http://jp.archive.ubuntu.com/ubuntu noble InRelease Get:4 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB] Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Get:6 http://jp.archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [344 kB] Get:7 http://jp.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [321 kB] Fetched 791 kB in 2s (365 kB/s) Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubeadm\u0026#39; The following packages will be upgraded: kubeadm 1 upgraded, 0 newly installed, 0 to remove and 32 not upgraded. Need to get 10.4 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubeadm 1.30.3-1.1 [10.4 MB] Fetched 10.4 MB in 0s (30.4 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110854 files and directories currently installed.) Preparing to unpack .../kubeadm_1.30.3-1.1_amd64.deb ... Unpacking kubeadm (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubeadm (1.30.3-1.1) ... Scanning processes... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-39-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. kubeadm set on hold. mao@k8s-worker-01:~$ \u0026ldquo;kubeadm upgrade\u0026quot;を実行する\n1 sudo kubeadm upgrade node 1 2 3 4 5 6 7 8 9 10 11 mao@k8s-worker-01:~$ sudo kubeadm upgrade node [upgrade] Reading configuration from the cluster... [upgrade] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [preflight] Running pre-flight checks [preflight] Skipping prepull. Not a control plane node. [upgrade] Skipping phase. Not a control plane node. [upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config1611631804/config.yaml [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [upgrade] The configuration for this node was successfully updated! [upgrade] Now you should go ahead and upgrade the kubelet package using your package manager. mao@k8s-worker-01:~$ Woker-Nodeをドレインする Control-Planeで下記のコマンドを実行する\n1 2 kubectl drain \u0026lt;node-to-drain\u0026gt; --ignore-daemonsets kubectl drain k8s-worker-01 --ignore-daemonsets 1 2 3 4 5 6 7 8 9 10 11 12 13 mao@k8s-control-plane-01:~$ kubectl drain k8s-worker-01 --ignore-daemonsets node/k8s-worker-01 cordoned Warning: ignoring DaemonSet-managed Pods: calico-system/calico-node-l2ll2, calico-system/csi-node-driver-2pzn6, kube-system/kube-proxy-rmxrw, metallb-system/speaker-2hccg evicting pod metallb-system/controller-86f5578878-xxpt6 evicting pod calico-system/calico-typha-5579b889c8-dc9zt evicting pod calico-apiserver/calico-apiserver-5f78767767-cnx87 evicting pod kube-system/coredns-7db6d8ff4d-m29qd pod/controller-86f5578878-xxpt6 evicted pod/calico-apiserver-5f78767767-cnx87 evicted pod/calico-typha-5579b889c8-dc9zt evicted pod/coredns-7db6d8ff4d-m29qd evicted node/k8s-worker-01 drained mao@k8s-control-plane-01:~$ 1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.3 k8s-worker-01 Ready,SchedulingDisabled \u0026lt;none\u0026gt; 42d v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-01:~$ kubeletとkubectlをアップデートする Worker-Nodeで実行する\n1 2 3 sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.3-*\u0026#39; kubectl=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 mao@k8s-worker-01:~$ sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.3-*\u0026#39; kubectl=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl Canceled hold on kubelet. Canceled hold on kubectl. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:4 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubelet\u0026#39; Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubectl\u0026#39; The following packages will be upgraded: kubectl kubelet 2 upgraded, 0 newly installed, 0 to remove and 30 not upgraded. Need to get 28.9 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubectl 1.30.3-1.1 [10.8 MB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubelet 1.30.3-1.1 [18.1 MB] Fetched 28.9 MB in 1s (53.4 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110854 files and directories currently installed.) Preparing to unpack .../kubectl_1.30.3-1.1_amd64.deb ... Unpacking kubectl (1.30.3-1.1) over (1.30.2-1.1) ... Preparing to unpack .../kubelet_1.30.3-1.1_amd64.deb ... Unpacking kubelet (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubectl (1.30.3-1.1) ... Setting up kubelet (1.30.3-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-39-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... systemctl restart kubelet.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. kubelet set on hold. kubectl set on hold. mao@k8s-worker-01:~$ リロードする\n1 2 sudo systemctl daemon-reload sudo systemctl restart kubelet nodeからpodを退避させているのを解除する \u0026ldquo;kubectl drain\u0026quot;を解除する\nControl-Planeで実行する\n1 2 kubectl uncordon \u0026lt;node-to-uncordon\u0026gt; kubectl uncordon k8s-worker-01 1 2 3 4 5 6 7 8 9 10 mao@k8s-control-plane-01:~$ kubectl uncordon k8s-worker-01 node/k8s-worker-01 uncordoned mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.3 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.3 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-01:~$ Woker-Nodeのバージョン確認 \u0026ldquo;kubectl get nodes\u0026quot;をControl-Planeで実行して\u0026quot;k8s-worker-01\u0026quot;のバージョンが\u0026quot;v1.30.3\u0026quot;にアップグレードされていることを確認\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.3 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.3 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.2 mao@k8s-control-plane-01:~$ これでWoker-Nodeのアップグレードは完了 他のWoker-Nodeもアップグレードする Woker-Node-02 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 mao@k8s-control-plane-01:~$ kubectl drain k8s-worker-02 --ignore-daemonsets node/k8s-worker-02 cordoned Warning: ignoring DaemonSet-managed Pods: calico-system/calico-node-28xrv, calico-system/csi-node-driver-jddlc, kube-system/kube-proxy-8hsdx, metallb-system/speaker-dpfpl evicting pod tigera-operator/tigera-operator-76ff79f7fd-hm5bk evicting pod calico-system/calico-typha-5579b889c8-pdlqz evicting pod metallb-system/controller-86f5578878-chzwd evicting pod kube-system/coredns-7db6d8ff4d-5kfxd evicting pod calico-apiserver/calico-apiserver-5f78767767-gjh5t evicting pod calico-system/calico-kube-controllers-5f5665469b-fbfh2 pod/controller-86f5578878-chzwd evicted pod/tigera-operator-76ff79f7fd-hm5bk evicted pod/calico-kube-controllers-5f5665469b-fbfh2 evicted pod/calico-apiserver-5f78767767-gjh5t evicted pod/coredns-7db6d8ff4d-5kfxd evicted pod/calico-typha-5579b889c8-pdlqz evicted node/k8s-worker-02 drained mao@k8s-control-plane-01:~$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 mao@k8s-worker-02:~$ sudo apt-mark unhold kubeadm \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubeadm=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubeadm Canceled hold on kubeadm. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubeadm\u0026#39; The following packages will be upgraded: kubeadm 1 upgraded, 0 newly installed, 0 to remove and 30 not upgraded. Need to get 10.4 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubeadm 1.30.3-1.1 [10.4 MB] Fetched 10.4 MB in 0s (31.1 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110852 files and directories currently installed.) Preparing to unpack .../kubeadm_1.30.3-1.1_amd64.deb ... Unpacking kubeadm (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubeadm (1.30.3-1.1) ... Scanning processes... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-39-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. kubeadm set on hold. mao@k8s-worker-02:~$ sudo kubeadm upgrade node [upgrade] Reading configuration from the cluster... [upgrade] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [preflight] Running pre-flight checks [preflight] Skipping prepull. Not a control plane node. [upgrade] Skipping phase. Not a control plane node. [upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config2769406187/config.yaml [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [upgrade] The configuration for this node was successfully updated! [upgrade] Now you should go ahead and upgrade the kubelet package using your package manager. mao@k8s-worker-02:~$ sudo apt-mark unhold kubelet kubectl \u0026amp;\u0026amp; \\ sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y kubelet=\u0026#39;1.30.3-*\u0026#39; kubectl=\u0026#39;1.30.3-*\u0026#39; \u0026amp;\u0026amp; \\ sudo apt-mark hold kubelet kubectl [sudo] password for mao: Canceled hold on kubelet. Canceled hold on kubectl. Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease Hit:2 http://jp.archive.ubuntu.com/ubuntu noble InRelease Get:3 http://jp.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB] Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:5 http://jp.archive.ubuntu.com/ubuntu noble-backports InRelease Get:6 http://jp.archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [344 kB] Get:7 http://jp.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [321 kB] Fetched 791 kB in 2s (469 kB/s) Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubelet\u0026#39; Selected version \u0026#39;1.30.3-1.1\u0026#39; (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for \u0026#39;kubectl\u0026#39; The following packages will be upgraded: kubectl kubelet 2 upgraded, 0 newly installed, 0 to remove and 31 not upgraded. Need to get 28.9 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubectl 1.30.3-1.1 [10.8 MB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubelet 1.30.3-1.1 [18.1 MB] Fetched 28.9 MB in 0s (57.8 MB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 110852 files and directories currently installed.) Preparing to unpack .../kubectl_1.30.3-1.1_amd64.deb ... Unpacking kubectl (1.30.3-1.1) over (1.30.2-1.1) ... Preparing to unpack .../kubelet_1.30.3-1.1_amd64.deb ... Unpacking kubelet (1.30.3-1.1) over (1.30.2-1.1) ... Setting up kubectl (1.30.3-1.1) ... Setting up kubelet (1.30.3-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 6.8.0-39-generic Diagnostics: The currently running kernel version is not the expected kernel version 6.8.0-40-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... systemctl restart kubelet.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. kubelet set on hold. kubectl set on hold. mao@k8s-worker-02:~$ sudo systemctl daemon-reload mao@k8s-worker-02:~$ sudo systemctl restart kubelet mao@k8s-worker-02:~$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.3 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.3 k8s-worker-02 Ready,SchedulingDisabled \u0026lt;none\u0026gt; 42d v1.30.3 mao@k8s-control-plane-01:~$ kubectl uncordon k8s-worker-02 node/k8s-worker-02 uncordoned mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.3 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.3 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.3 mao@k8s-control-plane-01:~$ これでWoker-Nodeのアップグレードは完了 クラスタのアップグレードが完了しました \u0026ldquo;v1.30.2\u0026quot;が\u0026quot;v1.30.3\u0026quot;へ\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 42d v1.30.3 k8s-control-plane-02 Ready control-plane 42d v1.30.3 k8s-control-plane-03 Ready control-plane 42d v1.30.3 k8s-worker-01 Ready \u0026lt;none\u0026gt; 42d v1.30.3 k8s-worker-02 Ready \u0026lt;none\u0026gt; 42d v1.30.3 mao@k8s-control-plane-01:~$ podも問題なく動作しているか確認\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 mao@k8s-control-plane-01:~$ kubectl get pod -A -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES calico-apiserver calico-apiserver-5f78767767-78qj9 1/1 Running 0 9m1s 10.128.36.245 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-apiserver calico-apiserver-5f78767767-9q5bf 1/1 Running 0 16m 10.128.204.140 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-kube-controllers-5f5665469b-qjbzw 1/1 Running 0 9m1s 10.128.36.244 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-26sbk 1/1 Running 4 (82m ago) 42d 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-28xrv 1/1 Running 4 (82m ago) 42d 192.168.10.43 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-dc87d 1/1 Running 4 (83m ago) 42d 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-g2ks8 1/1 Running 4 (82m ago) 42d 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-node-l2ll2 1/1 Running 4 (82m ago) 42d 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-typha-5579b889c8-b4jzx 1/1 Running 0 12m 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-typha-5579b889c8-fxdx8 1/1 Running 0 6m27s 192.168.10.43 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system calico-typha-5579b889c8-wrmhq 1/1 Running 0 25m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-2pzn6 2/2 Running 8 (82m ago) 42d 10.128.36.239 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-8b6ts 2/2 Running 8 (83m ago) 42d 10.128.251.148 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-9w86p 2/2 Running 8 (82m ago) 42d 10.128.204.138 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-cljz8 2/2 Running 8 (36h ago) 42d 10.128.194.202 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; calico-system csi-node-driver-jddlc 2/2 Running 8 (82m ago) 42d 10.128.118.105 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system coredns-7db6d8ff4d-cbws9 1/1 Running 0 9m1s 10.128.36.242 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system coredns-7db6d8ff4d-lx728 1/1 Running 0 16m 10.128.204.139 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system etcd-k8s-control-plane-01 1/1 Running 34 (83m ago) 42d 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system etcd-k8s-control-plane-02 1/1 Running 0 34m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system etcd-k8s-control-plane-03 1/1 Running 0 27m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-apiserver-k8s-control-plane-01 1/1 Running 0 44m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-apiserver-k8s-control-plane-02 1/1 Running 0 33m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-apiserver-k8s-control-plane-03 1/1 Running 0 27m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-controller-manager-k8s-control-plane-01 1/1 Running 0 44m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-controller-manager-k8s-control-plane-02 1/1 Running 0 33m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-controller-manager-k8s-control-plane-03 1/1 Running 0 26m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-8hsdx 1/1 Running 0 26m 192.168.10.43 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-ldpnm 1/1 Running 0 26m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-qb2c4 1/1 Running 0 26m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-rmxrw 1/1 Running 0 26m 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-proxy-v6k6r 1/1 Running 0 26m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-scheduler-k8s-control-plane-01 1/1 Running 0 43m 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-scheduler-k8s-control-plane-02 1/1 Running 0 32m 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; kube-system kube-scheduler-k8s-control-plane-03 1/1 Running 0 26m 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system controller-86f5578878-9gqs5 1/1 Running 0 9m1s 10.128.36.243 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-2hccg 1/1 Running 8 (81m ago) 42d 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-cjz7j 1/1 Running 8 (81m ago) 42d 192.168.10.44 k8s-control-plane-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-dpfpl 1/1 Running 8 (81m ago) 42d 192.168.10.43 k8s-worker-02 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-gg452 1/1 Running 8 (81m ago) 42d 192.168.10.46 k8s-control-plane-03 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; metallb-system speaker-zj5x9 1/1 Running 8 (81m ago) 42d 192.168.10.41 k8s-control-plane-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; tigera-operator tigera-operator-76ff79f7fd-rq76h 1/1 Running 0 9m1s 192.168.10.42 k8s-worker-01 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; mao@k8s-control-plane-01:~$ 参考URL https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/ https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/ https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/upgrading-linux-nodes/ https://goodbyegangster.hatenablog.com/entry/2021/01/19/205313 ","date":"2024-08-13T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-upgrade-v1-30-3/","title":"Kubernetesのクラスタをv1.30.2からv1.30.3へアップグレードをする"},{"content":"環境 Kubernetes 1.30.2 Helm v3.15.3 kubernetes-dashboard-7.5.0 Helmの導入 参考URL\nhttps://helm.sh/ja/docs/intro/install/ https://github.com/helm/helm/releases https://qiita.com/loftkun/items/bcfe1f205cde4d74a384 今回はバイナリを使用する方法でインストールします\nバイナリをダウンロードします\n1 wget https://get.helm.sh/helm-v3.15.3-linux-amd64.tar.gz 1 2 3 4 5 6 7 8 9 10 11 12 13 mao@k8s-control-plane-01:~$ wget https://get.helm.sh/helm-v3.15.3-linux-amd64.tar.gz --2024-08-08 09:11:52-- https://get.helm.sh/helm-v3.15.3-linux-amd64.tar.gz Resolving get.helm.sh (get.helm.sh)... 152.199.39.108, 2606:2800:247:1cb7:261b:1f9c:2074:3c Connecting to get.helm.sh (get.helm.sh)|152.199.39.108|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 16626424 (16M) [application/x-tar] Saving to: ‘helm-v3.15.3-linux-amd64.tar.gz’ helm-v3.15.3-linux-am 100%[======================\u0026gt;] 15.86M 102MB/s in 0.2s 2024-08-08 09:11:52 (102 MB/s) - ‘helm-v3.15.3-linux-amd64.tar.gz’ saved [16626424/16626424] mao@k8s-control-plane-01:~$ 圧縮されているので展開します\n展開するとバイナリファイルが出てきます\n1 tar -zxvf helm-v3.15.3-linux-amd64.tar.gz 1 2 3 4 5 6 mao@k8s-control-plane-01:~$ tar -zxvf helm-v3.15.3-linux-amd64.tar.gz linux-amd64/ linux-amd64/helm linux-amd64/README.md linux-amd64/LICENSE mao@k8s-control-plane-01:~$ バイナリファイルを移動します\n1 sudo mv linux-amd64/helm /usr/local/bin/helm バイナリを移動したらバージョンを確認してみます\n1 helm version 1 2 3 mao@k8s-control-plane-01:~$ helm version version.BuildInfo{Version:\u0026#34;v3.15.3\u0026#34;, GitCommit:\u0026#34;3bb50bbbdd9c946ba9989fbe4fb4104766302a64\u0026#34;, GitTreeState:\u0026#34;clean\u0026#34;, GoVersion:\u0026#34;go1.22.5\u0026#34;} mao@k8s-control-plane-01:~$ \u0026ldquo;v3.15.3\u0026quot;と表示されたのでこれでインストールは完了しました\nアップデートする際はこのバイナリを新しいものに置き換えます\nアンインストール方法 1 2 3 $ which helm /usr/local/bin/helm $ sudo rm /usr/local/bin/helm Helmを使用してKubernetes-dashboardをデプロイする 参考URL\nhttps://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/ https://artifacthub.io/packages/helm/k8s-dashboard/kubernetes-dashboard https://qiita.com/loftkun/items/bcfe1f205cde4d74a38 Helmにリポジトリを追加 1 helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/ 1 2 3 mao@k8s-control-plane-01:~$ helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/ \u0026#34;kubernetes-dashboard\u0026#34; has been added to your repositories mao@k8s-control-plane-01:~$ 追加されているか確認をする\n1 2 3 4 mao@k8s-control-plane-01:~$ helm repo list NAME URL kubernetes-dashboard https://kubernetes.github.io/dashboard/ mao@k8s-control-plane-01:~$ Helmを使用してデプロイ 公式サイトに記載にある手順でデプロイします\n1 helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 mao@k8s-control-plane-01:~$ helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard Release \u0026#34;kubernetes-dashboard\u0026#34; does not exist. Installing it now. NAME: kubernetes-dashboard LAST DEPLOYED: Thu Aug 8 12:22:06 2024 NAMESPACE: kubernetes-dashboard STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: ************************************************************************************************* *** PLEASE BE PATIENT: Kubernetes Dashboard may need a few minutes to get up and become ready *** ************************************************************************************************* Congratulations! You have just installed Kubernetes Dashboard in your cluster. To access Dashboard run: kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard-kong-proxy 8443:443 NOTE: In case port-forward command does not work, make sure that kong service name is correct. Check the services in Kubernetes Dashboard namespace using: kubectl -n kubernetes-dashboard get svc Dashboard will be available at: https://localhost:8443 mao@k8s-control-plane-01:~$ サービスアカウントの作成 参考URL\nhttps://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md マニフェストファイルを作成して以下の内容を書き込みます\ndashboard-adminuser.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 apiVersion: v1 kind: ServiceAccount metadata: name: admin-user namespace: kubernetes-dashboard --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: admin-user namespace: kubernetes-dashboard デプロイします\n1 kubectl apply -f dashboard-adminuser.yaml 1 2 3 4 mao@k8s-control-plane-01:~$ kubectl apply -f dashboard-adminuser.yaml serviceaccount/admin-user created clusterrolebinding.rbac.authorization.k8s.io/admin-user created mao@k8s-control-plane-01:~$ 外部からアクセスできるようにする ポートフォワードをする\n1 kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard-kong-proxy 8443:443 Ctrl+Cでポートフォワード停止\nログイン用のトークンを確認 外部からアクセスできるようにしてブラウザにIPアドレスを入力して表示するとログインを求められるので、\n以下の方法でログイン用のトークンを発行します\n1 kubectl -n kubernetes-dashboard create token admin-user 1 2 3 mao@k8s-control-plane-01:~$ kubectl -n kubernetes-dashboard create token admin-user eyJhbGciOiJSUzI1NiIsImtpZCI6ImtRS1lrek1TRUE3LXRCeEVpU2hJRDhEUGhWT0lzeG1QM1lwNjBHbkl1MlkifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNzIzMTI0NjUyLCJpYXQiOjE3MjMxMjEwNTIsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwianRpIjoiNDk4ZTg0YjctNTFmZi00NWRjLTgxODYtOGVhODMwMTY4NDdkIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsInNlcnZpY2VhY2NvdW50Ijp7Im5hbWUiOiJhZG1pbi11c2VyIiwidWlkIjoiNDczMzgwYWMtZWM3Yi00ZTBkLThiOTMtN2E1YjdkNDkzMTI5In19LCJuYmYiOjE3MjMxMjEwNTIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDphZG1pbi11c2VyIn0.WoGFlfvaUxchL1RqUP60GPEr7Q4laleBmixc-XyeZp48R4dVaZILC9eCsgDGJZbc3Q9_uh9ynVBle40lOLShwvsvbMcvrecNT-konG2HHCaUnMRV_nZeS7qaT5pYZCwKskW9AkdRsXS4dDKY7Wlj6jEStryM0OcYNdPm0JtxtXX6ejK5qA0wl6zPpcqLmnPHqMhRcUw0gHuIu9AL9cAoTT7sDUPBZSBwE1P5MS-eNAQx05xfNkUkilL1kNN-gnE-PX7u79uCXTi8lhoYlO48fQMOoo6MIGsqw9QBZcfx-uFGolwrWa3KmyQuaOAc8pghUjipGtV1KHSkEA1TDjx69g mao@k8s-control-plane-01:~$ ログイン画面 サービスアカウントの削除 1 kubectl delete -f dashboard-adminuser.yaml 1 2 3 4 mao@k8s-control-plane-01:~$ kubectl delete -f dashboard-adminuser.yaml serviceaccount \u0026#34;admin-user\u0026#34; deleted clusterrolebinding.rbac.authorization.k8s.io \u0026#34;admin-user\u0026#34; deleted mao@k8s-control-plane-01:~$ 上記の方法でサービスアカウントを削除すれば以下はやらなくても大丈夫 1 2 kubectl -n kubernetes-dashboard delete serviceaccount admin-user kubectl -n kubernetes-dashboard delete clusterrolebinding admin-user ダッシュボードを削除 1 helm uninstall release_name -n release_namespace 1 helm uninstall kubernetes-dashboard -n kubernetes-dashboard 1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~/k8s$ helm ls -A NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION kubernetes-dashboard kubernetes-dashboard 1 2024-08-08 10:09:12.228703023 +0000 UTC deployed kubernetes-dashboard-7.5.0 mao@k8s-control-plane-01:~/k8s$ helm uninstall kubernetes-dashboard -n kubernetes-dashboard release \u0026#34;kubernetes-dashboard\u0026#34; uninstalled mao@k8s-control-plane-01:~/k8s$ helm ls -A NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION mao@k8s-control-plane-01:~/k8s$ 無事削除されたことを確認 1 2 3 mao@k8s-control-plane-01:~$ kubectl -n kubernetes-dashboard get svc No resources found in kubernetes-dashboard namespace. mao@k8s-control-plane-01:~$ kubernetes-dashboardをクラスタ外からLoadBlancerを使いアクセスできるようにする 今までの方法だと外部からアクセスする際にポートフォワードをしなければならず、大変＋LoadBalancerがあるので、LoadBalancerでIPアドレスを割り当てて、そこに外部からアクセスできるようにしました。\n設定ファイルのダウンロード\nhttps://github.com/kubernetes/dashboard/blob/master/charts/kubernetes-dashboard/values.yaml 1 wget https://raw.githubusercontent.com/kubernetes/dashboard/master/charts/kubernetes-dashboard/values.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 mao@k8s-control-plane-01:~$ wget https://raw.githubusercontent.com/kubernetes/dashboard/master/charts/kubernetes-dashboard/values.yaml --2024-08-09 23:37:44-- https://raw.githubusercontent.com/kubernetes/dashboard/master/charts/kubernetes-dashboard/values.yaml Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.108.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 12820 (13K) [text/plain] Saving to: ‘values.yaml’ values.yaml 100%[===================\u0026gt;] 12.52K --.-KB/s in 0s 2024-08-09 23:37:44 (128 MB/s) - ‘values.yaml’ saved [12820/12820] mao@k8s-control-plane-01:~$ デプロイする\n\u0026ldquo;-f values.yaml\u0026quot;でファイルを指定する https://github.com/kubernetes/dashboard/tree/master/charts/kubernetes-dashboard 1 helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard -f values.yaml \u0026ldquo;values.yaml\u0026quot;ファイルの\u0026quot;kong.proxy.type\u0026quot;が\u0026quot;ClusterIP\u0026quot;になっていたので\u0026quot;LoadBalancer\u0026quot;へと変更したらクラスタ外からでもアクセスできるようになった 普通にIPアドレスのみでアクセスすると\u0026quot;400 Bad Request\u0026quot;になるので\u0026quot;https\u0026quot;を付けたらアクセスできた 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 kong: enabled: true ## Configuration reference: https://docs.konghq.com/gateway/3.6.x/reference/configuration env: dns_order: LAST,A,CNAME,AAAA,SRV plugins: \u0026#39;off\u0026#39; nginx_worker_processes: 1 ingressController: enabled: false dblessConfig: configMap: kong-dbless-config proxy: #type: ClusterIP type: LoadBalancer #loadBalancerIP: 192.168.10.57 http: enabled: false \u0026ldquo;EXTERNAL-IP\u0026quot;が割り当てられている 1 2 3 4 5 6 7 8 9 mao@k8s-control-plane-01:~$ kubectl -n kubernetes-dashboard get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes-dashboard-api ClusterIP 10.111.166.219 \u0026lt;none\u0026gt; 8000/TCP 10m kubernetes-dashboard-auth ClusterIP 10.110.8.100 \u0026lt;none\u0026gt; 8000/TCP 10m kubernetes-dashboard-kong-manager NodePort 10.99.108.147 \u0026lt;none\u0026gt; 8002:31994/TCP,8445:31824/TCP 10m kubernetes-dashboard-kong-proxy LoadBalancer 10.103.61.118 192.168.10.55 443:32210/TCP 10m kubernetes-dashboard-metrics-scraper ClusterIP 10.111.179.107 \u0026lt;none\u0026gt; 8000/TCP 10m kubernetes-dashboard-web ClusterIP 10.100.158.51 \u0026lt;none\u0026gt; 8000/TCP 10m mao@k8s-control-plane-01:~$ リソースの削除 サービスアカウントの削除 1 kubectl delete -f dashboard-adminuser.yaml 1 2 3 4 mao@k8s-control-plane-01:~$ kubectl delete -f dashboard-adminuser.yaml serviceaccount \u0026#34;admin-user\u0026#34; deleted clusterrolebinding.rbac.authorization.k8s.io \u0026#34;admin-user\u0026#34; deleted mao@k8s-control-plane-01:~$ 上記の方法でサービスアカウントを削除すれば以下はやらなくても大丈夫 1 2 kubectl -n kubernetes-dashboard delete serviceaccount admin-user kubectl -n kubernetes-dashboard delete clusterrolebinding admin-user ダッシュボードを削除 1 helm uninstall release_name -n release_namespace 1 helm uninstall kubernetes-dashboard -n kubernetes-dashboard 1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~/k8s$ helm ls -A NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION kubernetes-dashboard kubernetes-dashboard 1 2024-08-08 10:09:12.228703023 +0000 UTC deployed kubernetes-dashboard-7.5.0 mao@k8s-control-plane-01:~/k8s$ helm uninstall kubernetes-dashboard -n kubernetes-dashboard release \u0026#34;kubernetes-dashboard\u0026#34; uninstalled mao@k8s-control-plane-01:~/k8s$ helm ls -A NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION mao@k8s-control-plane-01:~/k8s$ 無事削除されたことを確認 1 2 3 mao@k8s-control-plane-01:~$ kubectl -n kubernetes-dashboard get svc No resources found in kubernetes-dashboard namespace. mao@k8s-control-plane-01:~$ ","date":"2024-08-12T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-dashboard/","title":"Kubernetesのダッシュボードをデプロイする＋LoadBalancerでIPアドレスを割り振る"},{"content":"環境 Kubernetes 1.30.2 containerd 1.7.18 Calico 3.28.0 MetalLB 0.14.5 IPアドレスのプール:192.168.10.55-192.168.10.58 背景 MetalLBを使用してtype:loadBalancerが使用できるようになったが、IPアドレスの指定方法がわからなかったので、調べて作業してみました。\n\u0026ldquo;このサービスにはこのIPアドレスを使用する\u0026quot;のように自分でIPアドレスを指定したい。\n元のマニフェストファイル nginx-test.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 10 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.27 ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: nginx-deployment-lb annotations: spec: type: LoadBalancer ports: - port: 80 targetPort: 80 selector: app: nginx 現状 普通にデプロイするとIPプールの中から空いているIPアドレスが付与される\n実行結果 1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl apply -f nginx-test.yaml deployment.apps/nginx-deployment created service/nginx-deployment-lb created mao@k8s-control-plane-01:~$ kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 38d nginx-deployment-lb LoadBalancer 10.96.182.105 192.168.10.55 80:30974/TCP 3s mao@k8s-control-plane-01:~$ 削除します 1 2 3 mao@k8s-control-plane-01:~$ kubectl delete -f nginx-test.yaml deployment.apps \u0026#34;nginx-deployment\u0026#34; deleted service \u0026#34;nginx-deployment-lb\u0026#34; deleted LoadBalancerでIPアドレスを指定する MetalLBを使用していると2つの方法があるようなので両方試してみます\nMetalLBのページでは\u0026quot;metadata\u0026quot;を使用した方法が推奨されている（spec.LoadBalancerIPはk8s apisで非推奨となる予定だからのようです）\nspec.loadBalancerIPに指定する \u0026ldquo;spec\u0026quot;に\u0026quot;loadBalancerIP:192.168.10.57\u0026quot;を追加してデプロイしてみます\nマニフェストファイル 1 2 3 4 5 6 7 8 9 10 11 12 13 --- apiVersion: v1 kind: Service metadata: name: nginx-deployment-lb spec: type: LoadBalancer + loadBalancerIP: 192.168.10.57 ports: - port: 80 targetPort: 80 selector: app: nginx 実行結果 1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl apply -f nginx-test.yaml deployment.apps/nginx-deployment created service/nginx-deployment-lb created mao@k8s-control-plane-01:~$ kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 38d nginx-deployment-lb LoadBalancer 10.111.232.189 192.168.10.57 80:31150/TCP 4s mao@k8s-control-plane-01:~$ しっかりと指定したIPアドレスがが付与されています\n削除します 1 2 3 mao@k8s-control-plane-01:~$ kubectl delete -f nginx-test.yaml deployment.apps \u0026#34;nginx-deployment\u0026#34; deleted service \u0026#34;nginx-deployment-lb\u0026#34; deleted metadata.annotationsに指定する マニフェストファイルを修正します\n\u0026ldquo;metadata\u0026quot;に\u0026quot;annotations:\u0026ldquo;と\u0026quot;metallb.universe.tf/loadBalancerIPs: 192.168.10.56\u0026quot;を追加する \u0026ldquo;loadBalancerIP: 192.168.10.57\u0026quot;を削除する 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 --- apiVersion: v1 kind: Service metadata: name: nginx-deployment-lb + annotations: + metallb.universe.tf/loadBalancerIPs: 192.168.10.56 spec: type: LoadBalancer - loadBalancerIP: 192.168.10.57 ports: - port: 80 targetPort: 80 selector: app: nginx 実行結果 1 2 3 4 5 6 7 8 mao@k8s-control-plane-01:~$ kubectl apply -f nginx-test.yaml deployment.apps/nginx-deployment created service/nginx-deployment-lb created mao@k8s-control-plane-01:~$ kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 38d nginx-deployment-lb LoadBalancer 10.110.178.62 192.168.10.56 80:32442/TCP 3s mao@k8s-control-plane-01:~$ マニフェストファイルで指定したIPアドレスが付与されている\n終わったら削除する\n1 2 3 4 mao@k8s-control-plane-01:~$ kubectl delete -f nginx-test.yaml deployment.apps \u0026#34;nginx-deployment\u0026#34; deleted service \u0026#34;nginx-deployment-lb\u0026#34; deleted mao@k8s-control-plane-01:~$ 参考URL https://metallb.universe.tf/usage/ https://cstoku.dev/posts/2018/k8sdojo-09/ https://qiita.com/suzuyui/items/8f53a80edf2b32d45be2 https://blog.framinal.life/entry/2020/04/16/022042 ","date":"2024-08-11T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-loadbalancer-ip/","title":"KubernetesのLoadBalancerでIPアドレスを指定する方法"},{"content":"環境 Ansible 2.16.9 Kubernetes 1.30.3 containerd 1.7.20 runC 1.1.13 cni-plugins 1.5.1 ファイル・ディレクトリ構成 1 2 3 4 5 6 . ├── ansible.cfg ├── host.yaml └── playbook_k8s.yaml 1 directory, 3 files ansible.cfg\n1 2 3 [defaults] # fingerprintを検証しない設定 host_key_checking = False host.yaml\n1 2 3 4 5 6 7 8 9 10 11 12 13 # YAML は ”---\u0026#34; から開始する --- # \u0026#34;all\u0026#34; グループの宣言 all: # \u0026#34;all\u0026#34; グループに含まれるホストに関する情報を定義する宣言 hosts: # 管理対象ノードの情報を定義する宣言 ansible-test-server: ansible_host: 192.168.10.18 ansible_user: mao ansible_password: mao ansible_ssh_private_key_file: /home/mao/ansible-test/ansible-ssh ansible_port: 22 playbook_k8s.yaml\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 # Ansible-playbook - name: k8s setup hosts: - all become: yes tasks: ##### 1 - name: ansible.builtin.get_url: dest: /home/mao/ url: https://github.com/containerd/containerd/releases/download/v1.7.20/containerd-1.7.20-linux-amd64.tar.gz - name: bin containerd ansible.builtin.unarchive: remote_src: true src: /home/mao/containerd-1.7.20-linux-amd64.tar.gz dest: /usr/local - name: ansible.builtin.get_url: dest: /home/mao/ url: https://raw.githubusercontent.com/containerd/containerd/main/containerd.service - name: ansible.builtin.copy: remote_src: true src: /home/mao/containerd.service dest: /etc/systemd/system/containerd.service - name: daemon reload ansible.builtin.systemd_service: daemon_reload: true - name: containerd enable ansible.builtin.systemd_service: name: containerd enabled: true ##### 2 - name: runC download ansible.builtin.get_url: dest: /home/mao/ url: https://github.com/opencontainers/runc/releases/download/v1.1.13/runc.amd64 - name: runC install ansible.builtin.command: cmd: install -m 755 runc.amd64 /usr/local/sbin/runc ##### 3 - name: CNI(Container Network Interface) plugin ansible.builtin.get_url: dest: /home/mao/ url: https://github.com/containernetworking/plugins/releases/download/v1.5.1/cni-plugins-linux-amd64-v1.5.1.tgz - name: make directory ansible.builtin.file: path: /opt/cni/bin state: directory - name: ansible.builtin.unarchive: remote_src: true src: /home/mao/cni-plugins-linux-amd64-v1.5.1.tgz dest: /opt/cni/bin ##### 4 - name: ansible.builtin.shell: cmd: | cat \u0026gt; /etc/modules-load.d/k8s.conf \u0026lt;\u0026lt; EOF overlay br_netfilter EOF - name: ansible.builtin.command: cmd: modprobe overlay - name: ansible.builtin.command: cmd: modprobe br_netfilter - name: ansible.builtin.shell: cmd: | cat \u0026gt; /etc/sysctl.d/k8s.conf \u0026lt;\u0026lt; EOF net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 EOF - name: ansible.builtin.command: cmd: sysctl --system ##### 5 - name: make directory containerd ansible.builtin.file: path: /etc/containerd state: directory - name: cpoy ansible.builtin.shell: cmd: sudo containerd config default | sudo tee /etc/containerd/config.toml - name: 1 ansible.builtin.lineinfile: dest: /etc/containerd/config.toml state: present backrefs: yes regexp: sandbox_image = \u0026#34;registry.k8s.io/pause:3.8\u0026#34; line: \u0026#39; sandbox_image = \u0026#34;registry.k8s.io/pause:3.9\u0026#34;\u0026#39; - name: 2 ansible.builtin.lineinfile: dest: /etc/containerd/config.toml state: present backrefs: yes regexp: SystemdCgroup = false line: \u0026#39; SystemdCgroup = true\u0026#39; - name: 3 ansible.builtin.service: name: containerd state: restarted ##### 6 - name: install ansible.builtin.apt: pkg: - apt-transport-https - ca-certificates - curl - gpg update_cache: yes state: present - name: ansible.builtin.shell: cmd: curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg - name: ansible.builtin.shell: cmd: echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list # kubeadm:1.30.3,kubectl:1.30.3,kubelet:1.30.3 - name: ansible.builtin.apt: pkg: - kubelet - kubeadm - kubectl update_cache: yes state: present 内容 上記playbookの中にある\u0026quot;#####\u0026ldquo;は内容ごとに分割しています。 ファイルを分けるのは後々やりたいと思っています。\n1 containerdのインストール 2 runCのインストール 3 CNI pluginのインストール 4 パラメータの設定 5 containerdの追加設定 6 kubeadm等のインストール 上記playbookとは別にIPアドレスの固定化、kubernetesクラスタへの参加は手動です。\nこちらも近いうちに自動化できるようにしようと思います。\nそうすれば自動でクラスタが構築できるようになります！\nあとはリポジトリの追加等、Ansibleモジュールをちゃんと利用した方法へ置き換えていこうと思います。\n","date":"2024-07-27T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/ansible-k8s/","title":"Ansibleを使ってKubernetesの初期設定をしてみた"},{"content":"Ansibleの拡張機能のインストール VScodeの拡張機能\u0026quot;redhat.ansible\u0026quot;があるのでインストールしておくと便利になる Ansibleのインストール 参考URL\nhttps://docs.ansible.com/ansible/latest/installation_guide/installation_distros.html#installing-ansible-on-ubuntu 1 2 3 sudo apt install software-properties-common sudo apt-add-repository --yes --update ppa:ansible/ansible sudo apt install ansible バージョンの確認\n1 2 ansible --version which ansible ファイル構造を見れるようにtreeをインストール 1 sudo apt install tree アドホック・コマンド フォルダの構造\n1 2 3 4 5 6 7 8 mao@ansible-server:~/ansible-test$ tree . ├── ansible-ssh ├── ansible.cfg └── host.yaml 1 directory, 3 files mao@ansible-server:~/ansible-test$ ファイルの中身 ansible-ssh\n実行されてsshで接続されると生成されるファイル ansible.cfg（設定ファイル）\n1 2 3 [defaults] # fingerprintを検証しない設定 host_key_checking = False host.yaml（インベントリーファイル）\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # YAML は ”---\u0026#34; から開始する --- # \u0026#34;all\u0026#34; グループの宣言 all: # \u0026#34;all\u0026#34; グループに含まれるホストに関する情報を定義する宣言 hosts: # 管理対象ノードの情報を定義する宣言 ansible-test-server: ansible_host: 192.168.10.10 ansible_user: mao ansible_password: mao ansible_ssh_private_key_file: /home/mao/ansible-test/ansible-ssh #ansible_python_interpreter: Python インタープリターの path ansible_port: 22 実行 下記コマンドを実行すると、それぞれのサーバーのストレージ容量を確認できる\n1 2 ansible all -i host.yaml -m ansible.builtin.command -a \u0026#34;df -h\u0026#34; ansible all -i hosts.yml -a \u0026#34;df -h\u0026#34; \u0026ldquo;-i\u0026quot;はインベントリーファイルの指定 \u0026ldquo;-m ansible.builtin.command\u0026quot;は短縮可能 実行結果\n1 2 3 4 5 6 7 8 9 10 mao@ansible-server:~/ansible-test$ ansible all -i host.yaml -m ansible.builtin.command -a \u0026#34;df -h\u0026#34; ansible-test-server | CHANGED | rc=0 \u0026gt;\u0026gt; Filesystem Size Used Avail Use% Mounted on tmpfs 795M 692K 794M 1% /run /dev/mapper/ubuntu--vg-ubuntu--lv 8.1G 2.5G 5.2G 32% / tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda2 1.7G 181M 1.5G 12% /boot tmpfs 795M 12K 795M 1% /run/user/1000 mao@ansible-server:~/ansible-test$ Ansible-playbookの実行 参考URL\nhttps://docs.ansible.com/ansible/latest/collections/index.html ファイル構造 1 2 3 4 5 6 7 8 9 mao@ansible-server:~/ansible-test$ tree . ├── ansible-ssh ├── ansible.cfg ├── host.yaml └── nginx_playbook.yaml 1 directory, 4 files mao@ansible-server:~/ansible-test$ \u0026ldquo;nginx_playbook.yaml\u0026quot;の中身\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # Ansible-playbook - name: setup nginx server hosts: - all become: yes tasks: - name: Install ansible.builtin.command: apt install -y nginx - name: Start ansible.builtin.service: name: nginx state: started - name: Enable nginx ansible.builtin.service: name: nginx #enabled: yes enabled: no #- name: nginx version #ansible.builtin.command: nginx -v #- name: status #ansible.builtin.command: systemctl status nginx 実行 nginxがインストールされます\n1 ansible-playbook -i host.yaml nginx_playbook.yaml --ask-become-pass \u0026ldquo;\u0026ndash;ask-become-pass\u0026rdquo;:sudoパスワードを求めるオプション 実行結果\n実行時にパスワードを求められるので入力します 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 mao@ansible-server:~/ansible-test$ ansible-playbook -i host.yaml nginx_playbook.yaml --ask-become-pass BECOME password: PLAY [setup nginx server] ******************************************************************** TASK [Gathering Facts] *********************************************************************** ok: [ansible-test-server] TASK [Install] ******************************************************************************* changed: [ansible-test-server] TASK [Start] ********************************************************************************* ok: [ansible-test-server] TASK [Enable nginx] ************************************************************************** ok: [ansible-test-server] PLAY RECAP *********************************************************************************** ansible-test-server : ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 mao@ansible-server:~/ansible-test$ 無事nginxが起動しています\n","date":"2024-07-15T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/ansible/","title":"Ansibleを少し触ってみた"},{"content":"開発環境 Proxmox 8.2.4 Ubuntu Server 24.04 LTS Kubernetes v1.30.2 HAProxyをセットアップする 1 2 sudo apt update sudo apt upgrade IPアドレスを固定する 99-config.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 network: version: 2 renderer: networkd ethernets: ens18: dhcp4: false addresses: - 192.168.10.45/24 routes: - to: default via: 192.168.10.1 nameservers: search: [] addresses: [192.168.10.1] ファイルを反映する\n1 2 sudo cp 99-config.yaml /etc/netplan/ sudo netplan apply 1 sudo chmod 600 /etc/netplan/99-config.yaml HAProxyをインストールする 1 sudo apt install haproxy 1 2 sudo mv /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.default sudo nano /etc/haproxy/haproxy.cfg 下記の通りに編集する\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 defaults timeout connect 10s timeout client 30s timeout server 30s frontend k8s bind *:6443 mode tcp #option tcplog default_backend k8s_backend backend k8s_backend balance roundrobin server k8s-control-plane-01 192.168.10.41:6443 check server k8s-control-plane-02 192.168.10.44:6443 check #server k8s-control-plane-01 \u0026lt;control node2のip\u0026gt;:6443 check 反映する\n1 sudo systemctl enable --now haproxy HA構成のクラスタを構築する Control-Plane-01で下記コマンドを実行する 1 sudo kubeadm init --control-plane-endpoint=192.168.10.45:6443 --pod-network-cidr=10.128.0.0/16 --upload-certs \u0026ldquo;\u0026ndash;control-plane-endpoint=\u0026lt;IPアドレス\u0026gt;:6443\u0026rdquo;:Control-PlaneのIPアドレスとAPIサーバーのポートを指定する \u0026ldquo;\u0026ndash;pod-network-cidr=10.128.0.0/16\u0026rdquo;:Pod間ネットワークの指定する \u0026ldquo;\u0026ndash;upload-certs\u0026rdquo;:Control-Plane、Worker-Node間で共有する証明書を暗号化する 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 mao@k8s-control-plane-01:~$ sudo kubeadm init --control-plane-endpoint=192.168.10.45:6443 --pod-network-cidr=10.128.0.0/16 --upload-certs [init] Using Kubernetes version: v1.30.2 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using \u0026#39;kubeadm config images pull\u0026#39; [certs] Using certificateDir folder \u0026#34;/etc/kubernetes/pki\u0026#34; [certs] Generating \u0026#34;ca\u0026#34; certificate and key [certs] Generating \u0026#34;apiserver\u0026#34; certificate and key [certs] apiserver serving cert is signed for DNS names [k8s-control-plane-01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.10.41 192.168.10.45] [certs] Generating \u0026#34;apiserver-kubelet-client\u0026#34; certificate and key [certs] Generating \u0026#34;front-proxy-ca\u0026#34; certificate and key [certs] Generating \u0026#34;front-proxy-client\u0026#34; certificate and key [certs] Generating \u0026#34;etcd/ca\u0026#34; certificate and key [certs] Generating \u0026#34;etcd/server\u0026#34; certificate and key [certs] etcd/server serving cert is signed for DNS names [k8s-control-plane-01 localhost] and IPs [192.168.10.41 127.0.0.1 ::1] [certs] Generating \u0026#34;etcd/peer\u0026#34; certificate and key [certs] etcd/peer serving cert is signed for DNS names [k8s-control-plane-01 localhost] and IPs [192.168.10.41 127.0.0.1 ::1] [certs] Generating \u0026#34;etcd/healthcheck-client\u0026#34; certificate and key [certs] Generating \u0026#34;apiserver-etcd-client\u0026#34; certificate and key [certs] Generating \u0026#34;sa\u0026#34; key and public key [kubeconfig] Using kubeconfig folder \u0026#34;/etc/kubernetes\u0026#34; [kubeconfig] Writing \u0026#34;admin.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;super-admin.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;kubelet.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;controller-manager.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;scheduler.conf\u0026#34; kubeconfig file [etcd] Creating static Pod manifest for local etcd in \u0026#34;/etc/kubernetes/manifests\u0026#34; [control-plane] Using manifest folder \u0026#34;/etc/kubernetes/manifests\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-apiserver\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-controller-manager\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-scheduler\u0026#34; [kubelet-start] Writing kubelet environment file with flags to file \u0026#34;/var/lib/kubelet/kubeadm-flags.env\u0026#34; [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [kubelet-start] Starting the kubelet [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \u0026#34;/etc/kubernetes/manifests\u0026#34; [kubelet-check] Waiting for a healthy kubelet. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 501.677267ms [api-check] Waiting for a healthy API server. This can take up to 4m0s [api-check] The API server is healthy after 5.012535655s [upload-config] Storing the configuration used in ConfigMap \u0026#34;kubeadm-config\u0026#34; in the \u0026#34;kube-system\u0026#34; Namespace [kubelet] Creating a ConfigMap \u0026#34;kubelet-config\u0026#34; in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Storing the certificates in Secret \u0026#34;kubeadm-certs\u0026#34; in the \u0026#34;kube-system\u0026#34; Namespace [upload-certs] Using certificate key: 46c4727ea4da0877fd6e152f0c5d4842837ea949909e9ccb83a5c2f7331f53a9 [mark-control-plane] Marking the node k8s-control-plane-01 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node k8s-control-plane-01 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] [bootstrap-token] Using token: h3eh17.f1j6k27nzi34hd0w [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the \u0026#34;cluster-info\u0026#34; ConfigMap in the \u0026#34;kube-public\u0026#34; namespace [kubelet-finalize] Updating \u0026#34;/etc/kubernetes/kubelet.conf\u0026#34; to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run \u0026#34;kubectl apply -f [podnetwork].yaml\u0026#34; with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of the control-plane node running the following command on each as root: kubeadm join 192.168.10.45:6443 --token h3eh17.f1j6k27nzi34hd0w \\ --discovery-token-ca-cert-hash sha256:6ee64840cf218d5f9ee05a5138e0f543bf6b2359f51ab3d13fde7405370ba7a7 \\ --control-plane --certificate-key 46c4727ea4da0877fd6e152f0c5d4842837ea949909e9ccb83a5c2f7331f53a9 Please note that the certificate-key gives access to cluster sensitive data, keep it secret! As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use \u0026#34;kubeadm init phase upload-certs --upload-certs\u0026#34; to reload certs afterward. Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.10.45:6443 --token h3eh17.f1j6k27nzi34hd0w \\ --discovery-token-ca-cert-hash sha256:6ee64840cf218d5f9ee05a5138e0f543bf6b2359f51ab3d13fde7405370ba7a7 mao@k8s-control-plane-01:~$ kubeadmコマンドを実行できるようにする 上記の実行結果に記載のある通り下記のコマンドを実行する\n1 2 3 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config クラスとにJoinするために必要な情報をメモ（記録）しておく 上記の実行結果に記載のある通りクラスタにJoinする際に必要なコマンドをメモしておく\nControl-PlaneがクラスタにJoinするために必要な情報\n1 2 3 kubeadm join 192.168.10.45:6443 --token h3eh17.f1j6k27nzi34hd0w \\ --discovery-token-ca-cert-hash sha256:6ee64840cf218d5f9ee05a5138e0f543bf6b2359f51ab3d13fde7405370ba7a7 \\ --control-plane --certificate-key 46c4727ea4da0877fd6e152f0c5d4842837ea949909e9ccb83a5c2f7331f53a9 Worker-NodeがクラスタにJoinするために必要な情報\n1 2 kubeadm join 192.168.10.45:6443 --token h3eh17.f1j6k27nzi34hd0w \\ --discovery-token-ca-cert-hash sha256:6ee64840cf218d5f9ee05a5138e0f543bf6b2359f51ab3d13fde7405370ba7a7 他のControl-PlaneをクラスタにJoinする 下記コマンドを実行する\n1 sudo kubeadm join 192.168.10.45:6443 --token h3eh17.f1j6k27nzi34hd0w --discovery-token-ca-cert-hash sha256:6ee64840cf218d5f9ee05a5138e0f543bf6b2359f51ab3d13fde7405370ba7a7 --control-plane --certificate-key 46c4727ea4da0877fd6e152f0c5d4842837ea949909e9ccb83a5c2f7331f53a9 Woker-NodeをクラスタにJoinする Worker-NodeをクラスタにJoinさせる、1つ目以外のWoker-Nodeも同じようにクラスタに参加させる\n1 sudo kubeadm join 192.168.10.45:6443 --token h3eh17.f1j6k27nzi34hd0w --discovery-token-ca-cert-hash sha256:6ee64840cf218d5f9ee05a5138e0f543bf6b2359f51ab3d13fde7405370ba7a7 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 mao@k8s-worker-01:~$ sudo kubeadm join 192.168.10.45:6443 --token h3eh17.f1j6k27nzi34hd0w --discovery-token-ca-cert-hash sha256:6ee64840cf218d5f9ee05a5138e0f543bf6b2359f51ab3d13fde7405370ba7a7 [sudo] password for mao: [preflight] Running pre-flight checks [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [kubelet-start] Writing kubelet environment file with flags to file \u0026#34;/var/lib/kubelet/kubeadm-flags.env\u0026#34; [kubelet-start] Starting the kubelet [kubelet-check] Waiting for a healthy kubelet. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 500.998945ms [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run \u0026#39;kubectl get nodes\u0026#39; on the control-plane to see this node join the cluster. mao@k8s-worker-01:~$ HA構成の確認のためにControl-Planeを1つダウンさせてみる Control-Planeを1つシャットダウンさせて擬似的にダウンさせてみる\n1 2 3 4 5 6 7 8 mao@k8s-control-plane-03:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 146m v1.30.2 k8s-control-plane-02 NotReady control-plane 140m v1.30.2 k8s-control-plane-03 Ready control-plane 10m v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 136m v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 127m v1.30.2 mao@k8s-control-plane-03:~$ 確認用のコマンドが問題なく実行され、Control-Planeが1つ\u0026quot;NotReady\u0026quot;になっているが、問題なくクラスタが稼働している\n参考 IPアドレスの構成 192.168.10.45,HAProxy 192.168.10.41,Control-Plane-01 192.168.10.44,Control-Plane-02 192.168.10.46,Control-Plane-03 192.168.10.42,Worker-Node-01 192.168.10.43,Worker-Node-02 参考URL https://kubernetes.io/ja/docs/setup/#production-environment https://kubernetes.io/ja/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/ https://kubernetes.io/ja/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ 感想 以上でHA構成のkubernetesの構築は完了しました\n公式ドキュメントを参考にしつつ進めてわからないところは検索したりして構築できました\nネットワーク部分に関しては構築前よりも詳しくなったと思います\n発展として、他にも\u0026quot;keepalive\u0026quot;や\u0026quot;kube-vip\u0026quot;を使用した構成もあるみたいなので、いずれ試してみようと思います\nあとkubernetes上に仮想マシンを構成する\u0026quot;kubevirt\u0026quot;や\u0026quot;openstack on kubernetes\u0026quot;を構築してみようと思います\n","date":"2024-07-15T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-on-proxmox-05/","title":"kubernetesをproxmox上に立ててみた（5）/HA構成"},{"content":"開発環境 Proxmox 8.2.4 Ubuntu Server 24.04 LTS Kubernetes v1.30.2 Worker-Nodeをクラスターから外す 先にControl-Planeですること 先にControl-Plane上でクラスターから外す準備をする\n1 2 kubectl drain k8s-worker-01 --ignore-daemonsets --delete-emptydir-data --force kubectl get node 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 mao@k8s-control-plane-01:~$ kubectl drain k8s-worker-01 --ignore-daemonsets --delete-emptydir-data --force node/k8s-worker-01 cordoned Warning: ignoring DaemonSet-managed Pods: calico-system/calico-node-h75vn, calico-system/csi-node-driver-5b5k4, kube-system/kube-proxy-wzvtf, metallb-system/speaker-xwb66 evicting pod metallb-system/controller-86f5578878-dz5zm pod/controller-86f5578878-dz5zm evicted node/k8s-worker-01 drained mao@k8s-control-plane-01:~$ kubectl get node NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 3d23h v1.30.2 k8s-worker-01 Ready,SchedulingDisabled \u0026lt;none\u0026gt; 3d22h v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 3d14h v1.30.2 mao@k8s-control-plane-01:~$ STATUSが\u0026quot;SchedulingDisabled\u0026quot;になったらOK\nWorker-Nodeでの作業 Control-Planeでの作業ができたら、次はWorker-Nodeで作業をする\n1 sudo kubeadm reset 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 mao@k8s-worker-01:~$ sudo kubeadm reset W0630 01:12:34.493241 63962 preflight.go:56] [reset] WARNING: Changes made to this host by \u0026#39;kubeadm init\u0026#39; or \u0026#39;kubeadm join\u0026#39; will be reverted. [reset] Are you sure you want to proceed? [y/N]: y [preflight] Running pre-flight checks W0630 01:12:36.338665 63962 removeetcdmember.go:106] [reset] No kubeadm config, using etcd pod spec to get data directory [reset] Deleted contents of the etcd data directory: /var/lib/etcd [reset] Stopping the kubelet service [reset] Unmounting mounted directories in \u0026#34;/var/lib/kubelet\u0026#34; [reset] Deleting contents of directories: [/etc/kubernetes/manifests /var/lib/kubelet /etc/kubernetes/pki] [reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/super-admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf] The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d The reset process does not reset or clean up iptables rules or IPVS tables. If you wish to reset iptables, you must do so manually by using the \u0026#34;iptables\u0026#34; command. If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar) to reset your system\u0026#39;s IPVS tables. The reset process does not clean your kubeconfig files and you must remove them manually. Please, check the contents of the $HOME/.kube/config file. Worker-Node上のcalicoを削除する 削除するネットワークインターフェースを確認する\n1 ip link 実行結果\n1 2 3 4 5 6 7 8 9 mao@k8s-worker-01:~$ ip link 1: lo: \u0026lt;LOOPBACK,UP,LOWER_UP\u0026gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: ens18: \u0026lt;BROADCAST,MULTICAST,UP,LOWER_UP\u0026gt; mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether bc:24:11:fe:35:a0 brd ff:ff:ff:ff:ff:ff altname enp0s18 10: vxlan.calico: \u0026lt;BROADCAST,MULTICAST,UP,LOWER_UP\u0026gt; mtu 1450 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/ether 66:bb:bb:d8:11:bf brd ff:ff:ff:ff:ff:ff mao@k8s-worker-01:~$ \u0026ldquo;vxlan.calico\u0026quot;を削除する\n1 sudo ip link delete vxlan.calico 最後にControl-PlaneからWorker-Nodeを削除する Control-Plane上で下記コマンドを実行する\n1 kubectl delete node k8s-worker-01 実行結果\n1 2 3 mao@k8s-control-plane-01:~$ kubectl delete node k8s-worker-01 node \u0026#34;k8s-worker-01\u0026#34; deleted mao@k8s-control-plane-01:~$ 削除されたことを確認する\n1 kubectl get node 実行結果\n1 2 3 4 5 mao@k8s-control-plane-01:~$ kubectl get node NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 3d23h v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 3d14h v1.30.2 mao@k8s-control-plane-01:~$ Control-Planeをクラスターから外す、クラスターを削除する Control-Plane上で、下記のコマンドを順に実行してクラスタをリセットする\n1 2 3 4 5 6 7 sudo kubeadm reset sudo rm -rf $HOME/.kube sudo systemctl daemon-reload \u0026amp;\u0026amp; systemctl restart kubelet sudo systemctl restart containerd sudo ip link sudo ip link delete vxlan.calico sudo ip link 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 mao@k8s-control-plane-01:~$ sudo kubeadm reset [reset] Reading configuration from the cluster... [reset] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; W0630 03:05:22.877860 124743 preflight.go:56] [reset] WARNING: Changes made to this host by \u0026#39;kubeadm init\u0026#39; or \u0026#39;kubeadm join\u0026#39; will be reverted. [reset] Are you sure you want to proceed? [y/N]: y [preflight] Running pre-flight checks [reset] Deleted contents of the etcd data directory: /var/lib/etcd [reset] Stopping the kubelet service [reset] Unmounting mounted directories in \u0026#34;/var/lib/kubelet\u0026#34; W0630 03:05:30.442255 124743 cleanupnode.go:106] [reset] Failed to remove containers: [failed to stop running pod 67f96751dedca872f742388cb86d92f0388de2eef490d813d380e706cb8c7424: output: E0630 03:05:29.277302 126592 remote_runtime.go:222] \u0026#34;StopPodSandbox from runtime service failed\u0026#34; err=\u0026#34;rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;67f96751dedca872f742388cb86d92f0388de2eef490d813d380e706cb8c7424\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; podSandboxID=\u0026#34;67f96751dedca872f742388cb86d92f0388de2eef490d813d380e706cb8c7424\u0026#34; time=\u0026#34;2024-06-30T03:05:29Z\u0026#34; level=fatal msg=\u0026#34;stopping the pod sandbox \\\u0026#34;67f96751dedca872f742388cb86d92f0388de2eef490d813d380e706cb8c7424\\\u0026#34;: rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;67f96751dedca872f742388cb86d92f0388de2eef490d813d380e706cb8c7424\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; : exit status 1, failed to stop running pod 9b4a0f41c7cafc48e51a68e89a0e9ad265fee646ef148a7ade366f90be956ee4: output: E0630 03:05:29.422959 126734 remote_runtime.go:222] \u0026#34;StopPodSandbox from runtime service failed\u0026#34; err=\u0026#34;rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;9b4a0f41c7cafc48e51a68e89a0e9ad265fee646ef148a7ade366f90be956ee4\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; podSandboxID=\u0026#34;9b4a0f41c7cafc48e51a68e89a0e9ad265fee646ef148a7ade366f90be956ee4\u0026#34; time=\u0026#34;2024-06-30T03:05:29Z\u0026#34; level=fatal msg=\u0026#34;stopping the pod sandbox \\\u0026#34;9b4a0f41c7cafc48e51a68e89a0e9ad265fee646ef148a7ade366f90be956ee4\\\u0026#34;: rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;9b4a0f41c7cafc48e51a68e89a0e9ad265fee646ef148a7ade366f90be956ee4\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; : exit status 1, failed to stop running pod 976d40faccfc6491f13fc990ae3c94ee0348faa04ab74968d05b5c5904c75e12: output: E0630 03:05:29.566837 126875 remote_runtime.go:222] \u0026#34;StopPodSandbox from runtime service failed\u0026#34; err=\u0026#34;rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;976d40faccfc6491f13fc990ae3c94ee0348faa04ab74968d05b5c5904c75e12\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; podSandboxID=\u0026#34;976d40faccfc6491f13fc990ae3c94ee0348faa04ab74968d05b5c5904c75e12\u0026#34; time=\u0026#34;2024-06-30T03:05:29Z\u0026#34; level=fatal msg=\u0026#34;stopping the pod sandbox \\\u0026#34;976d40faccfc6491f13fc990ae3c94ee0348faa04ab74968d05b5c5904c75e12\\\u0026#34;: rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;976d40faccfc6491f13fc990ae3c94ee0348faa04ab74968d05b5c5904c75e12\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; : exit status 1, failed to stop running pod 28a5274b17af2a1a78e80d18c48a8732826f7a020fef31901eec76fcaec91fc5: output: E0630 03:05:29.713590 127015 remote_runtime.go:222] \u0026#34;StopPodSandbox from runtime service failed\u0026#34; err=\u0026#34;rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;28a5274b17af2a1a78e80d18c48a8732826f7a020fef31901eec76fcaec91fc5\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; podSandboxID=\u0026#34;28a5274b17af2a1a78e80d18c48a8732826f7a020fef31901eec76fcaec91fc5\u0026#34; time=\u0026#34;2024-06-30T03:05:29Z\u0026#34; level=fatal msg=\u0026#34;stopping the pod sandbox \\\u0026#34;28a5274b17af2a1a78e80d18c48a8732826f7a020fef31901eec76fcaec91fc5\\\u0026#34;: rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;28a5274b17af2a1a78e80d18c48a8732826f7a020fef31901eec76fcaec91fc5\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; : exit status 1, failed to stop running pod 2a4ddac4e1d2d11184fb82e8283250c0756a73c23fe053d12edf9a4f037eb976: output: E0630 03:05:29.858624 127156 remote_runtime.go:222] \u0026#34;StopPodSandbox from runtime service failed\u0026#34; err=\u0026#34;rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;2a4ddac4e1d2d11184fb82e8283250c0756a73c23fe053d12edf9a4f037eb976\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; podSandboxID=\u0026#34;2a4ddac4e1d2d11184fb82e8283250c0756a73c23fe053d12edf9a4f037eb976\u0026#34; time=\u0026#34;2024-06-30T03:05:29Z\u0026#34; level=fatal msg=\u0026#34;stopping the pod sandbox \\\u0026#34;2a4ddac4e1d2d11184fb82e8283250c0756a73c23fe053d12edf9a4f037eb976\\\u0026#34;: rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;2a4ddac4e1d2d11184fb82e8283250c0756a73c23fe053d12edf9a4f037eb976\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; : exit status 1, failed to stop running pod 70174a13942604fd21e17d0425064f463e35c347067301b9051206c088123112: output: E0630 03:05:30.010265 127297 remote_runtime.go:222] \u0026#34;StopPodSandbox from runtime service failed\u0026#34; err=\u0026#34;rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;70174a13942604fd21e17d0425064f463e35c347067301b9051206c088123112\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; podSandboxID=\u0026#34;70174a13942604fd21e17d0425064f463e35c347067301b9051206c088123112\u0026#34; time=\u0026#34;2024-06-30T03:05:30Z\u0026#34; level=fatal msg=\u0026#34;stopping the pod sandbox \\\u0026#34;70174a13942604fd21e17d0425064f463e35c347067301b9051206c088123112\\\u0026#34;: rpc error: code = Unknown desc = failed to destroy network for sandbox \\\u0026#34;70174a13942604fd21e17d0425064f463e35c347067301b9051206c088123112\\\u0026#34;: plugin type=\\\u0026#34;calico\\\u0026#34; failed (delete): error getting ClusterInformation: Get \\\u0026#34;https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default\\\u0026#34;: dial tcp 10.96.0.1:443: connect: connection refused\u0026#34; : exit status 1] [reset] Deleting contents of directories: [/etc/kubernetes/manifests /var/lib/kubelet /etc/kubernetes/pki] [reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/super-admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf] The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d The reset process does not reset or clean up iptables rules or IPVS tables. If you wish to reset iptables, you must do so manually by using the \u0026#34;iptables\u0026#34; command. If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar) to reset your system\u0026#39;s IPVS tables. The reset process does not clean your kubeconfig files and you must remove them manually. Please, check the contents of the $HOME/.kube/config file. mao@k8s-control-plane-01:~$ 1 2 3 4 5 6 7 mao@k8s-control-plane-01:~$ sudo systemctl daemon-reload \u0026amp;\u0026amp; systemctl restart kubelet ==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ==== Authentication is required to restart \u0026#39;kubelet.service\u0026#39;. Authenticating as: mao Password: ==== AUTHENTICATION COMPLETE ==== mao@k8s-control-plane-01:~$ 1 2 3 4 5 6 7 8 9 mao@k8s-control-plane-01:~$ sudo ip link 1: lo: \u0026lt;LOOPBACK,UP,LOWER_UP\u0026gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: ens18: \u0026lt;BROADCAST,MULTICAST,UP,LOWER_UP\u0026gt; mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether bc:24:11:7f:08:e4 brd ff:ff:ff:ff:ff:ff altname enp0s18 9: vxlan.calico: \u0026lt;BROADCAST,MULTICAST,UP,LOWER_UP\u0026gt; mtu 1450 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/ether 66:ca:40:c1:cc:6f brd ff:ff:ff:ff:ff:ff mao@k8s-control-plane-01:~$ 1 2 3 4 5 6 7 mao@k8s-control-plane-01:~$ sudo ip link 1: lo: \u0026lt;LOOPBACK,UP,LOWER_UP\u0026gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: ens18: \u0026lt;BROADCAST,MULTICAST,UP,LOWER_UP\u0026gt; mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether bc:24:11:7f:08:e4 brd ff:ff:ff:ff:ff:ff altname enp0s18 mao@k8s-control-plane-01:~$ これでクラスターをリセットができました\n","date":"2024-07-10T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-on-proxmox-04/","title":"kubernetesをproxmox上に立ててみた（4）/クラスターを壊す"},{"content":"開発環境 Proxmox 8.2.4 Ubuntu Server 24.04 LTS Kubernetes v1.30.2 LoadBalancer（MetalLB）の設定をする kubernetesのクラスターの外部からIPアドレスでアクセスするための設定をする ロードバランサー（MetalLB）を使用して外部からアクセスできるようにする マニフェストファイルの\u0026quot;type\u0026quot;に\u0026quot;LoadBalancer\u0026quot;を指定できるようになる MetalLB を実行する 今回はMetalLBを使用する\nhttps://metallb.universe.tf/installation/ ARPの設定をする 1 1 2 3 kubectl get configmap kube-proxy -n kube-system -o yaml | \\ sed -e \u0026#34;s/strictARP: false/strictARP: true/\u0026#34; | \\ kubectl diff -f - -n kube-system 2 1 2 3 kubectl get configmap kube-proxy -n kube-system -o yaml | \\ sed -e \u0026#34;s/strictARP: false/strictARP: true/\u0026#34; | \\ kubectl apply -f - -n kube-system 実行結果 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 mao@k8s-control-plane-01:~$ kubectl get configmap kube-proxy -n kube-system -o yaml | \\ sed -e \u0026#34;s/strictARP: false/strictARP: true/\u0026#34; | \\ kubectl diff -f - -n kube-system diff -u -N /tmp/LIVE-1007559117/v1.ConfigMap.kube-system.kube-proxy /tmp/MERGED-3311346621/v1.ConfigMap.kube-system.kube-proxy --- /tmp/LIVE-1007559117/v1.ConfigMap.kube-system.kube-proxy 2024-06-26 12:33:11.717730136 +0000 +++ /tmp/MERGED-3311346621/v1.ConfigMap.kube-system.kube-proxy 2024-06-26 12:33:11.718730159 +0000 @@ -37,7 +37,7 @@ excludeCIDRs: null minSyncPeriod: 0s scheduler: \u0026#34;\u0026#34; - strictARP: false + strictARP: true syncPeriod: 0s tcpFinTimeout: 0s tcpTimeout: 0s mao@k8s-control-plane-01:~$ kubectl get configmap kube-proxy -n kube-system -o yaml | \\ sed -e \u0026#34;s/strictARP: false/strictARP: true/\u0026#34; | \\ kubectl apply -f - -n kube-system Warning: resource configmaps/kube-proxy is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. configmap/kube-proxy configured mao@k8s-control-plane-01:~$ MetalLBの実行をする 参考URL\nhttps://blog.ntnx.jp/entry/2024/02/14/025309 1 2 wget https://raw.githubusercontent.com/metallb/metallb/v0.14.5/config/manifests/metallb-native.yaml kubectl apply -f metallb-native.yaml 確認をする\n1 kubectl get pod -n metallb-system 実行結果\n1 2 3 4 5 6 7 mao@k8s-control-plane-01:~$ kubectl get pod -n metallb-system NAME READY STATUS RESTARTS AGE controller-86f5578878-dz5zm 1/1 Running 0 43s speaker-mbvpk 1/1 Running 0 43s speaker-rx75f 1/1 Running 0 43s speaker-xwb66 1/1 Running 0 43s mao@k8s-control-plane-01:~$ 払い出せるIPアドレスの範囲を設定する\n下記の部分で範囲を設定する 1 2 3 spec: addresses: - 192.168.10.55-192.168.10.60 1 1 2 3 4 5 6 7 8 9 10 cat \u0026lt;\u0026lt; EOF \u0026gt; ippool.yaml apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: default namespace: metallb-system spec: addresses: - 192.168.10.55-192.168.10.60 EOF 2 1 2 3 4 5 6 7 8 9 10 cat \u0026lt;\u0026lt; EOF \u0026gt; l2adv.yaml apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: default namespace: metallb-system spec: ipAddressPools: - default EOF 設定を実行する\n1 kubectl apply -f ippool.yaml -f l2adv.yaml 実行結果\n1 2 3 mao@k8s-control-plane-01:~$ kubectl apply -f ippool.yaml -f l2adv.yaml ipaddresspool.metallb.io/default created l2advertisement.metallb.io/default created マニフェストファイルを実行後、外部IPアドレスが設定されているか確認する マニフェストファイル\nnginx-lb.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 10 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.27 ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: nginx-deployment-lb spec: type: LoadBalancer #type: ClusterIP ports: - port: 80 targetPort: 80 selector: app: nginx \u0026ldquo;type: LoadBalancer\u0026quot;にするとLoadBalancerからIPアドレスが払い出されてクラスタ外からアクセスできるようになる マニフェストファイルを実行してデプロイする\n1 kubectl apply -f nginx-lb.yaml 確認するためのコマンド\n1 2 3 kubectl get svc or kubectl get service 実行結果\n1 2 3 4 5 mao@k8s-control-plane-01:~$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 \u0026lt;none\u0026gt; 443/TCP 25h nginx-deployment-lb LoadBalancer 10.98.208.100 192.168.10.45 80:32762/TCP 58s mao@k8s-control-plane-01:~$ \u0026ldquo;nginx-deployment-lb\u0026quot;の\u0026quot;TYPE\u0026quot;に\u0026quot;LoadBalancer\u0026quot;が指定されており\u0026quot;EXTERNAL-IP\u0026rdquo;（外部IPアドレス）が割り当てられている\nこのIPアドレスにアクセスすると実際のコンテナにアクセスできる\n","date":"2024-07-07T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-on-proxmox-03/","title":"kubernetesをproxmox上に立ててみた（3）/LoadBalancerの設定"},{"content":"開発環境 Proxmox 8.2.4 Ubuntu Server 24.04 LTS Kubernetes v1.30.2 Control-Plane（Master-Node）の設定をする kubeadm init を実行する Control-Planeにするマシン上で実行する\n1 2 sudo kubeadm init --apiserver-advertise-address=Control-PlaneのIPアドレス --pod-network-cidr=10.128.0.0/16 sudo kubeadm init --apiserver-advertise-address=192.168.10.41 --pod-network-cidr=10.128.0.0/16 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 mao@k8s-control-plane-01:~$ sudo kubeadm init --apiserver-advertise-address=192.168.10.41 --pod-network-cidr=10.128.0.0/16 [init] Using Kubernetes version: v1.30.2 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using \u0026#39;kubeadm config images pull\u0026#39; [certs] Using certificateDir folder \u0026#34;/etc/kubernetes/pki\u0026#34; [certs] Generating \u0026#34;ca\u0026#34; certificate and key [certs] Generating \u0026#34;apiserver\u0026#34; certificate and key [certs] apiserver serving cert is signed for DNS names [k8s-control-plane-01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.10.41] [certs] Generating \u0026#34;apiserver-kubelet-client\u0026#34; certificate and key [certs] Generating \u0026#34;front-proxy-ca\u0026#34; certificate and key [certs] Generating \u0026#34;front-proxy-client\u0026#34; certificate and key [certs] Generating \u0026#34;etcd/ca\u0026#34; certificate and key [certs] Generating \u0026#34;etcd/server\u0026#34; certificate and key [certs] etcd/server serving cert is signed for DNS names [k8s-control-plane-01 localhost] and IPs [192.168.10.41 127.0.0.1 ::1] [certs] Generating \u0026#34;etcd/peer\u0026#34; certificate and key [certs] etcd/peer serving cert is signed for DNS names [k8s-control-plane-01 localhost] and IPs [192.168.10.41 127.0.0.1 ::1] [certs] Generating \u0026#34;etcd/healthcheck-client\u0026#34; certificate and key [certs] Generating \u0026#34;apiserver-etcd-client\u0026#34; certificate and key [certs] Generating \u0026#34;sa\u0026#34; key and public key [kubeconfig] Using kubeconfig folder \u0026#34;/etc/kubernetes\u0026#34; [kubeconfig] Writing \u0026#34;admin.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;super-admin.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;kubelet.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;controller-manager.conf\u0026#34; kubeconfig file [kubeconfig] Writing \u0026#34;scheduler.conf\u0026#34; kubeconfig file [etcd] Creating static Pod manifest for local etcd in \u0026#34;/etc/kubernetes/manifests\u0026#34; [control-plane] Using manifest folder \u0026#34;/etc/kubernetes/manifests\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-apiserver\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-controller-manager\u0026#34; [control-plane] Creating static Pod manifest for \u0026#34;kube-scheduler\u0026#34; [kubelet-start] Writing kubelet environment file with flags to file \u0026#34;/var/lib/kubelet/kubeadm-flags.env\u0026#34; [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [kubelet-start] Starting the kubelet [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \u0026#34;/etc/kubernetes/manifests\u0026#34; [kubelet-check] Waiting for a healthy kubelet. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 500.625027ms [api-check] Waiting for a healthy API server. This can take up to 4m0s [api-check] The API server is healthy after 3.50086825s [upload-config] Storing the configuration used in ConfigMap \u0026#34;kubeadm-config\u0026#34; in the \u0026#34;kube-system\u0026#34; Namespace [kubelet] Creating a ConfigMap \u0026#34;kubelet-config\u0026#34; in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node k8s-control-plane-01 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node k8s-control-plane-01 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] [bootstrap-token] Using token: evbpii.dp8y5hcfqkv9jn4n [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the \u0026#34;cluster-info\u0026#34; ConfigMap in the \u0026#34;kube-public\u0026#34; namespace [kubelet-finalize] Updating \u0026#34;/etc/kubernetes/kubelet.conf\u0026#34; to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run \u0026#34;kubectl apply -f [podnetwork].yaml\u0026#34; with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.10.41:6443 --token evbpii.dp8y5hcfqkv9jn4n \\ --discovery-token-ca-cert-hash sha256:dd7a24f7fcd7aeea509476025652a8a1aee32e9e8d5f54ec48de16345eb1a425 mao@k8s-control-plane-01:~$ 実行結果にも表示されている通り、下記のコマンドを実行する\n1 2 3 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Calicoを実行する（Pod間ネットワーク） 参考URL\nhttps://docs.tigera.io/calico/latest/getting-started/kubernetes/quickstart tigera-operator.yaml\n1 2 wget https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/tigera-operator.yaml kubectl create -f tigera-operator.yaml custom-resources.yaml\n1 wget https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/custom-resources.yaml ダウンロードしたcustom-resources.yamlを編集する\nkubeadm init で指定した引数\u0026ndash;pod-network-cidrと同じものへ変更する 1 2 - cidr: 192.168.0.0/16 + cidr: 10.128.0.0/16 実行する\n1 kubectl apply -f custom-resources.yaml Nodeを確認する 1 kubectl get nodes -o wide 実行結果\n1 2 3 4 mao@k8s-control-plane-01:~$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-control-plane-01 Ready control-plane 32m v1.30.2 192.168.10.41 \u0026lt;none\u0026gt; Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18 mao@k8s-control-plane-01:~$ Worker-Nodeの設定をする Worker-NodeをJoinする Control-Planeでkubeadm initを実行した際に表示されたコマンドを実行する\n1 sudo kubeadm join 192.168.10.41:6443 --token evbpii.dp8y5hcfqkv9jn4n --discovery-token-ca-cert-hash sha256:dd7a24f7fcd7aeea509476025652a8a1aee32e9e8d5f54ec48de16345eb1a425 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 mao@k8s-worker-01:~$ sudo kubeadm join 192.168.10.41:6443 --token evbpii.dp8y5hcfqkv9jn4n --discovery-to ken-ca-cert-hash sha256:dd7a24f7fcd7aeea509476025652a8a1aee32e9e8d5f54ec48de16345eb1a425 [preflight] Running pre-flight checks [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with \u0026#39;kubectl -n kube-system get cm kubeadm-config -o yaml\u0026#39; [kubelet-start] Writing kubelet configuration to file \u0026#34;/var/lib/kubelet/config.yaml\u0026#34; [kubelet-start] Writing kubelet environment file with flags to file \u0026#34;/var/lib/kubelet/kubeadm-flags.env\u0026#34; [kubelet-start] Starting the kubelet [kubelet-check] Waiting for a healthy kubelet. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 501.168161ms [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run \u0026#39;kubectl get nodes\u0026#39; on the control-plane to see this node join the cluster. mao@k8s-worker-01:~$ Worker-NodeがJoinされているか、Control-Planeで確認する\n1 kubectl get node 実行結果\n1 2 3 4 5 mao@k8s-control-plane-01:~$ kubectl get node NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 63m v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 36s v1.30.2 mao@k8s-control-plane-01:~$ 複数のWorker-NodeをJoinする 基本手順は1つ目のWorker-Nodeと同じように実行するとJoinできる\n1 sudo kubeadm join 192.168.10.41:6443 --token evbpii.dp8y5hcfqkv9jn4n --discovery-token-ca-cert-hash sha256:dd7a24f7fcd7aeea509476025652a8a1aee32e9e8d5f54ec48de16345eb1a425 実行したらControl-PlanでNodeを確認してみる\n1 2 3 4 5 6 mao@k8s-control-plane-01:~$ kubectl get node NAME STATUS ROLES AGE VERSION k8s-control-plane-01 Ready control-plane 9h v1.30.2 k8s-worker-01 Ready \u0026lt;none\u0026gt; 8h v1.30.2 k8s-worker-02 Ready \u0026lt;none\u0026gt; 83s v1.30.2 mao@k8s-control-plane-01:~$ 無事にWorker-NodeがJoinされている\n","date":"2024-07-06T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-on-proxmox-02/","title":"kubernetesをproxmox上に立ててみた（2）/Control-Plane・Worker-Nodeの設定"},{"content":" k8sを勉強してみようと思い、デプロイ等の操作は書籍で触っていたら構築もしてみたくなったので、Proxmox上に仮想マシンを作成してk8sを構築してみた\n開発環境 Proxmox 8.2.4 Ubuntu Server 24.04 LTS Kubernetes v1.30.2 構成 Proxmox上に以下6つの仮想マシンを立てました\nhaproxy-01 control-plane-node-01 control-plane-node-02 control-plane-node-03 worker-node-01 worker-node-02 Control-Plane（Master-Node）とWorker-Node両方で実行する setup パッケージの更新をする\n1 2 sudo apt update sudo apt upgrade 必要なソフトウェアをインストールする\n1 sudo apt install nano 公式の手順にそって実行する\nhttps://kubernetes.io/ja/docs/setup/production-environment/ Swapをオフにする swapを止めます\n1 sudo swapoff -a 設定ファイルを書き換えて永続的にswapをオフにする\n1 sudo nano /etc/fstab 編集内容\n1 2 - /swap.img none swap sw 0 0 + #/swap.img none swap sw 0 0 swapがオフになっているか確認する\n1 free -h 実行結果\n1 2 3 4 mao@k8s-control-plane-01:~$ free -h total used free shared buff/cache available Mem: 7.8Gi 510Mi 7.2Gi 704Ki 248Mi 7.3Gi Swap: 0B 0B 0B IPアドレスを固定IPアドレスにする ネットワークのデバイスを確認します\n1 ip address 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 mao@k8s-control-plane-01:~$ ip address 1: lo: \u0026lt;LOOPBACK,UP,LOWER_UP\u0026gt; mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever 2: ens18: \u0026lt;BROADCAST,MULTICAST,UP,LOWER_UP\u0026gt; mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether bc:24:11:7f:08:e4 brd ff:ff:ff:ff:ff:ff altname enp0s18 inet 192.168.10.10/24 metric 100 brd 192.168.10.255 scope global dynamic ens18 valid_lft 85953sec preferred_lft 85953sec inet6 fe80::be24:11ff:fe7f:8e4/64 scope link valid_lft forever preferred_lft forever mao@k8s-control-plane-01:~$ netplanファイルを作成します\nファイル名：99-config.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 network: version: 2 renderer: networkd ethernets: ens18: dhcp4: false addresses: - 192.168.10.41/24 routes: - to: default via: 192.168.10.1 nameservers: search: [] addresses: [192.168.10.1] netplanファイルをコピーして適用します\nssh等で接続している場合は、IPアドレスが変わるので接続が切れます、\n再度固定にしたIPアドレスに変更すれば接続できます 1 2 3 sudo cp 99-config.yaml /etc/netplan/ sudo netplan apply sudo chmod 600 /etc/netplan/99-config.yaml containerdをインストールする 公式手順に従ってインストール\nhttps://github.com/containerd/containerd/blob/main/docs/getting-started.md Option 1: From the official binaries containerd 1.7.18 順にコマンドを実行する\n1 2 wget https://github.com/containerd/containerd/releases/download/v1.7.18/containerd-1.7.18-linux-amd64.tar.gz sudo tar Cxzvf /usr/local containerd-1.7.18-linux-amd64.tar.gz 1 sudo wget https://raw.githubusercontent.com/containerd/containerd/main/containerd.service -O /etc/systemd/system/containerd.service systemctlをリロードし、containerdを有効にする\n1 2 sudo systemctl daemon-reload sudo systemctl enable --now containerd runCをインストールする 1 sudo wget https://github.com/opencontainers/runc/releases/download/v1.1.13/runc.amd64 1 sudo install -m 755 runc.amd64 /usr/local/sbin/runc CNI(Container Network Interface) pluginをインストールする 1 sudo wget https://github.com/containernetworking/plugins/releases/download/v1.5.1/cni-plugins-linux-amd64-v1.5.1.tgz 1 2 sudo mkdir -p /opt/cni/bin sudo tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.5.1.tgz IPv4フォワーディングの設定をする 以下のコマンドを順に実行する\n1 1 cat \u0026lt;\u0026lt;EOF | sudo tee /etc/modules-load.d/k8s.conf 実行結果\n1 2 3 4 5 6 7 mao@k8s-control-plane-01:~$ cat \u0026lt;\u0026lt;EOF | sudo tee /etc/modules-load.d/k8s.conf \u0026gt; overlay \u0026gt; br_netfilter \u0026gt; EOF overlay br_netfilter mao@k8s-control-plane-01:~$ 1 2 sudo modprobe overlay sudo modprobe br_netfilter 2 1 cat \u0026lt;\u0026lt;EOF | sudo tee /etc/sysctl.d/k8s.conf 実行結果\n1 2 3 4 5 6 7 8 9 mao@k8s-control-plane-01:~$ cat \u0026lt;\u0026lt;EOF | sudo tee /etc/sysctl.d/k8s.conf \u0026gt; net.bridge.bridge-nf-call-iptables = 1 \u0026gt; net.bridge.bridge-nf-call-ip6tables = 1 \u0026gt; net.ipv4.ip_forward = 1 \u0026gt; EOF net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 mao@k8s-control-plane-01:~$ 1 sudo sysctl --system 3 1 2 lsmod | grep br_netfilter lsmod | grep overlay 実行結果\n1 2 3 4 5 6 mao@k8s-control-plane-01:~$ lsmod | grep br_netfilter br_netfilter 32768 0 bridge 421888 1 br_netfilter mao@k8s-control-plane-01:~$ lsmod | grep overlay overlay 212992 0 mao@k8s-control-plane-01:~$ 4 1 sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward 実行結果\n1 2 3 4 5 mao@k8s-control-plane-01:~$ sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 mao@k8s-control-plane-01:~$ systemd cgroup の設定をする 参考URL\nhttps://kubernetes.io/ja/docs/concepts/architecture/cgroups/ https://kubernetes.io/ja/docs/concepts/architecture/cgroups/#check-cgroup-version https://sogo.dev/posts/2022/12/kubernetes-ubuntu22.04-cgroup-systemd 1 stat -fc %T /sys/fs/cgroup/ cgroup v2では、\u0026ldquo;cgroup2fs\u0026quot;と出力されます。 cgroup v1では、\u0026ldquo;tmpfs\u0026quot;と出力されます。 ディレクトリを作成する\n1 sudo mkdir /etc/containerd 以下のコマンドで、デフォルトのコンフィグを作成できます。\n1 sudo containerd config default | sudo tee /etc/containerd/config.toml 実行結果\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 mao@k8s-control-plane-01:~$ sudo containerd config default | sudo tee /etc/containerd/config.toml disabled_plugins = [] imports = [] oom_score = 0 plugin_dir = \u0026#34;\u0026#34; required_plugins = [] root = \u0026#34;/var/lib/containerd\u0026#34; state = \u0026#34;/run/containerd\u0026#34; temp = \u0026#34;\u0026#34; version = 2 [cgroup] path = \u0026#34;\u0026#34; [debug] address = \u0026#34;\u0026#34; format = \u0026#34;\u0026#34; gid = 0 level = \u0026#34;\u0026#34; uid = 0 [grpc] address = \u0026#34;/run/containerd/containerd.sock\u0026#34; gid = 0 max_recv_message_size = 16777216 max_send_message_size = 16777216 tcp_address = \u0026#34;\u0026#34; tcp_tls_ca = \u0026#34;\u0026#34; tcp_tls_cert = \u0026#34;\u0026#34; tcp_tls_key = \u0026#34;\u0026#34; uid = 0 [metrics] address = \u0026#34;\u0026#34; grpc_histogram = false [plugins] [plugins.\u0026#34;io.containerd.gc.v1.scheduler\u0026#34;] deletion_threshold = 0 mutation_threshold = 100 pause_threshold = 0.02 schedule_delay = \u0026#34;0s\u0026#34; startup_delay = \u0026#34;100ms\u0026#34; [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;] cdi_spec_dirs = [\u0026#34;/etc/cdi\u0026#34;, \u0026#34;/var/run/cdi\u0026#34;] device_ownership_from_security_context = false disable_apparmor = false disable_cgroup = false disable_hugetlb_controller = true disable_proc_mount = false disable_tcp_service = true drain_exec_sync_io_timeout = \u0026#34;0s\u0026#34; enable_cdi = false enable_selinux = false enable_tls_streaming = false enable_unprivileged_icmp = false enable_unprivileged_ports = false ignore_deprecation_warnings = [] ignore_image_defined_volumes = false image_pull_progress_timeout = \u0026#34;5m0s\u0026#34; image_pull_with_sync_fs = false max_concurrent_downloads = 3 max_container_log_line_size = 16384 netns_mounts_under_state_dir = false restrict_oom_score_adj = false sandbox_image = \u0026#34;registry.k8s.io/pause:3.8\u0026#34; selinux_category_range = 1024 stats_collect_period = 10 stream_idle_timeout = \u0026#34;4h0m0s\u0026#34; stream_server_address = \u0026#34;127.0.0.1\u0026#34; stream_server_port = \u0026#34;0\u0026#34; systemd_cgroup = false tolerate_missing_hugetlb_controller = true unset_seccomp_profile = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.cni] bin_dir = \u0026#34;/opt/cni/bin\u0026#34; conf_dir = \u0026#34;/etc/cni/net.d\u0026#34; conf_template = \u0026#34;\u0026#34; ip_pref = \u0026#34;\u0026#34; max_conf_num = 1 setup_serially = false [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd] default_runtime_name = \u0026#34;runc\u0026#34; disable_snapshot_annotations = true discard_unpacked_layers = false ignore_blockio_not_enabled_errors = false ignore_rdt_not_enabled_errors = false no_pivot = false snapshotter = \u0026#34;overlayfs\u0026#34; [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd.default_runtime] base_runtime_spec = \u0026#34;\u0026#34; cni_conf_dir = \u0026#34;\u0026#34; cni_max_conf_num = 0 container_annotations = [] pod_annotations = [] privileged_without_host_devices = false privileged_without_host_devices_all_devices_allowed = false runtime_engine = \u0026#34;\u0026#34; runtime_path = \u0026#34;\u0026#34; runtime_root = \u0026#34;\u0026#34; runtime_type = \u0026#34;\u0026#34; sandbox_mode = \u0026#34;\u0026#34; snapshotter = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd.default_runtime.options] [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd.runtimes] [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd.runtimes.runc] base_runtime_spec = \u0026#34;\u0026#34; cni_conf_dir = \u0026#34;\u0026#34; cni_max_conf_num = 0 container_annotations = [] pod_annotations = [] privileged_without_host_devices = false privileged_without_host_devices_all_devices_allowed = false runtime_engine = \u0026#34;\u0026#34; runtime_path = \u0026#34;\u0026#34; runtime_root = \u0026#34;\u0026#34; runtime_type = \u0026#34;io.containerd.runc.v2\u0026#34; sandbox_mode = \u0026#34;podsandbox\u0026#34; snapshotter = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd.runtimes.runc.options] BinaryName = \u0026#34;\u0026#34; CriuImagePath = \u0026#34;\u0026#34; CriuPath = \u0026#34;\u0026#34; CriuWorkPath = \u0026#34;\u0026#34; IoGid = 0 IoUid = 0 NoNewKeyring = false NoPivotRoot = false Root = \u0026#34;\u0026#34; ShimCgroup = \u0026#34;\u0026#34; SystemdCgroup = false [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd.untrusted_workload_runtime] base_runtime_spec = \u0026#34;\u0026#34; cni_conf_dir = \u0026#34;\u0026#34; cni_max_conf_num = 0 container_annotations = [] pod_annotations = [] privileged_without_host_devices = false privileged_without_host_devices_all_devices_allowed = false runtime_engine = \u0026#34;\u0026#34; runtime_path = \u0026#34;\u0026#34; runtime_root = \u0026#34;\u0026#34; runtime_type = \u0026#34;\u0026#34; sandbox_mode = \u0026#34;\u0026#34; snapshotter = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd.untrusted_workload_runtime.options] [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.image_decryption] key_model = \u0026#34;node\u0026#34; [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.registry] config_path = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.registry.auths] [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.registry.configs] [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.registry.headers] [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.registry.mirrors] [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.x509_key_pair_streaming] tls_cert_file = \u0026#34;\u0026#34; tls_key_file = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.internal.v1.opt\u0026#34;] path = \u0026#34;/opt/containerd\u0026#34; [plugins.\u0026#34;io.containerd.internal.v1.restart\u0026#34;] interval = \u0026#34;10s\u0026#34; [plugins.\u0026#34;io.containerd.internal.v1.tracing\u0026#34;] [plugins.\u0026#34;io.containerd.metadata.v1.bolt\u0026#34;] content_sharing_policy = \u0026#34;shared\u0026#34; [plugins.\u0026#34;io.containerd.monitor.v1.cgroups\u0026#34;] no_prometheus = false [plugins.\u0026#34;io.containerd.nri.v1.nri\u0026#34;] disable = true disable_connections = false plugin_config_path = \u0026#34;/etc/nri/conf.d\u0026#34; plugin_path = \u0026#34;/opt/nri/plugins\u0026#34; plugin_registration_timeout = \u0026#34;5s\u0026#34; plugin_request_timeout = \u0026#34;2s\u0026#34; socket_path = \u0026#34;/var/run/nri/nri.sock\u0026#34; [plugins.\u0026#34;io.containerd.runtime.v1.linux\u0026#34;] no_shim = false runtime = \u0026#34;runc\u0026#34; runtime_root = \u0026#34;\u0026#34; shim = \u0026#34;containerd-shim\u0026#34; shim_debug = false [plugins.\u0026#34;io.containerd.runtime.v2.task\u0026#34;] platforms = [\u0026#34;linux/amd64\u0026#34;] sched_core = false [plugins.\u0026#34;io.containerd.service.v1.diff-service\u0026#34;] default = [\u0026#34;walking\u0026#34;] [plugins.\u0026#34;io.containerd.service.v1.tasks-service\u0026#34;] blockio_config_file = \u0026#34;\u0026#34; rdt_config_file = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.snapshotter.v1.aufs\u0026#34;] root_path = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.snapshotter.v1.blockfile\u0026#34;] fs_type = \u0026#34;\u0026#34; mount_options = [] root_path = \u0026#34;\u0026#34; scratch_file = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.snapshotter.v1.btrfs\u0026#34;] root_path = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.snapshotter.v1.devmapper\u0026#34;] async_remove = false base_image_size = \u0026#34;\u0026#34; discard_blocks = false fs_options = \u0026#34;\u0026#34; fs_type = \u0026#34;\u0026#34; pool_name = \u0026#34;\u0026#34; root_path = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.snapshotter.v1.native\u0026#34;] root_path = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.snapshotter.v1.overlayfs\u0026#34;] mount_options = [] root_path = \u0026#34;\u0026#34; sync_remove = false upperdir_label = false [plugins.\u0026#34;io.containerd.snapshotter.v1.zfs\u0026#34;] root_path = \u0026#34;\u0026#34; [plugins.\u0026#34;io.containerd.tracing.processor.v1.otlp\u0026#34;] [plugins.\u0026#34;io.containerd.transfer.v1.local\u0026#34;] config_path = \u0026#34;\u0026#34; max_concurrent_downloads = 3 max_concurrent_uploaded_layers = 3 [[plugins.\u0026#34;io.containerd.transfer.v1.local\u0026#34;.unpack_config]] differ = \u0026#34;\u0026#34; platform = \u0026#34;linux/amd64\u0026#34; snapshotter = \u0026#34;overlayfs\u0026#34; [proxy_plugins] [stream_processors] [stream_processors.\u0026#34;io.containerd.ocicrypt.decoder.v1.tar\u0026#34;] accepts = [\u0026#34;application/vnd.oci.image.layer.v1.tar+encrypted\u0026#34;] args = [\u0026#34;--decryption-keys-path\u0026#34;, \u0026#34;/etc/containerd/ocicrypt/keys\u0026#34;] env = [\u0026#34;OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf\u0026#34;] path = \u0026#34;ctd-decoder\u0026#34; returns = \u0026#34;application/vnd.oci.image.layer.v1.tar\u0026#34; [stream_processors.\u0026#34;io.containerd.ocicrypt.decoder.v1.tar.gzip\u0026#34;] accepts = [\u0026#34;application/vnd.oci.image.layer.v1.tar+gzip+encrypted\u0026#34;] args = [\u0026#34;--decryption-keys-path\u0026#34;, \u0026#34;/etc/containerd/ocicrypt/keys\u0026#34;] env = [\u0026#34;OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf\u0026#34;] path = \u0026#34;ctd-decoder\u0026#34; returns = \u0026#34;application/vnd.oci.image.layer.v1.tar+gzip\u0026#34; [timeouts] \u0026#34;io.containerd.timeout.bolt.open\u0026#34; = \u0026#34;0s\u0026#34; \u0026#34;io.containerd.timeout.metrics.shimstats\u0026#34; = \u0026#34;2s\u0026#34; \u0026#34;io.containerd.timeout.shim.cleanup\u0026#34; = \u0026#34;5s\u0026#34; \u0026#34;io.containerd.timeout.shim.load\u0026#34; = \u0026#34;5s\u0026#34; \u0026#34;io.containerd.timeout.shim.shutdown\u0026#34; = \u0026#34;3s\u0026#34; \u0026#34;io.containerd.timeout.task.state\u0026#34; = \u0026#34;2s\u0026#34; [ttrpc] address = \u0026#34;\u0026#34; gid = 0 uid = 0 mao@k8s-control-plane-01:~$ 設定ファイルを編集する\n1 sudo nano /etc/containerd/config.toml 以下の2箇所を編集する\n1 2 3 4 5 6 7 [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;] - sandbox_image = \u0026#34;registry.k8s.io/pause:3.6\u0026#34; + sandbox_image = \u0026#34;registry.k8s.io/pause:3.9\u0026#34; ... [plugins.\u0026#34;io.containerd.grpc.v1.cri\u0026#34;.containerd.runtimes.runc.options] - SystemdCgroup = false + SystemdCgroup = true containerdを再起動する\n1 sudo systemctl restart containerd kubeadm/kubelet/kubectl をインストールする 参考URL\nhttps://kubernetes.io/ja/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ 以下のコマンドを順番に実行する\n1 sudo apt install apt-transport-https ca-certificates curl gpg 1 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg 1 echo \u0026#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /\u0026#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list 一度アップデートした後にインストールする\n1 2 sudo apt update sudo apt install kubelet kubeadm kubectl バージョンを固定する\n1 sudo apt-mark hold kubelet kubeadm kubectl 実行結果\n1 2 3 4 5 mao@k8s-control-plane-01:~$ sudo apt-mark hold kubelet kubeadm kubectl kubelet set on hold. kubeadm set on hold. kubectl set on hold. mao@k8s-control-plane-01:~$ バージョン固定の解除コマンド\n1 2 sudo apt-mark showhold sudo apt-mark unhold \u0026lt;パッケージ名\u0026gt; ","date":"2024-07-03T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/kubernetes-on-proxmox-01/","title":"kubernetesをproxmox上に立ててみた（1）"},{"content":"環境 Proxmox VE 8.2.4 x86_64 アップグレード前 メモリ16GB 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 root@pve:~# neofetch .://:` `://:. root@pve `hMMMMMMd/ /dMMMMMMh` -------- `sMMMMMMMd: :mMMMMMMMs` OS: Proxmox VE 8.2.4 x86_64 `-/+oo+/:`.yMMMMMMMh- -hMMMMMMMy.`:/+oo+/-` Kernel: 6.8.4-2-pve `:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:` Uptime: 33 days, 20 hours, 14 mins `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/` Packages: 852 (dpkg) ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/. Shell: bash 5.2.15 .+ooooooo+-`oNMMMMNo`-+ooooooo+. Terminal: /dev/pts/0 -+ooooooo/.`sMMs`./ooooooo+- CPU: AMD Ryzen 7 5700G with Radeon Graphi :oooooooo/`..`/oooooooo: GPU: AMD ATI Radeon Vega Series / Radeon :oooooooo/`..`/oooooooo: Memory: 1858MiB / 13837MiB -+ooooooo/.`sMMs`./ooooooo+- .+ooooooo+-`oNMMMMNo`-+ooooooo+. ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/. `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/` `:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:` `-/+oo+/:`.yMMMMMMMh- -hMMMMMMMy.`:/+oo+/-` `sMMMMMMMm: :dMMMMMMMs` `hMMMMMMd/ /dMMMMMMh` `://:` `://:` root@pve:~# アップグレード後 メモリ64GB 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 root@pve:~# neofetch .://:` `://:. root@pve `hMMMMMMd/ /dMMMMMMh` -------- `sMMMMMMMd: :mMMMMMMMs` OS: Proxmox VE 8.2.4 x86_64 `-/+oo+/:`.yMMMMMMMh- -hMMMMMMMy.`:/+oo+/-` Kernel: 6.8.8-1-pve `:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:` Uptime: 1 min `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/` Packages: 852 (dpkg) ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/. Shell: bash 5.2.15 .+ooooooo+-`oNMMMMNo`-+ooooooo+. Terminal: /dev/pts/0 -+ooooooo/.`sMMs`./ooooooo+- CPU: AMD Ryzen 7 5700G with Radeon Graphics (16) @ 4.673GHz :oooooooo/`..`/oooooooo: GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile Series :oooooooo/`..`/oooooooo: Memory: 1508MiB / 60133MiB -+ooooooo/.`sMMs`./ooooooo+- .+ooooooo+-`oNMMMMNo`-+ooooooo+. ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/. `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/` `:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:` `-/+oo+/:`.yMMMMMMMh- -hMMMMMMMy.`:/+oo+/-` `sMMMMMMMm: :dMMMMMMMs` `hMMMMMMd/ /dMMMMMMh` `://:` `://:` root@pve:~# ","date":"2024-06-22T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/deskminix300-memory-upgrade/","title":"ProxmoxがインストールされているDeskminix300のメモリを増設する"},{"content":"環境 Ubuntu 24.04 Zbbix 7.0 Ubuntu MySQL Nginx 手順 このページの通りにインストールをしていく(\u0026ldquo;https://www.zabbix.com/jp/download?zabbix=7.0\u0026os_distribution=ubuntu\u0026os_version=24.04\u0026components=server_frontend_agent\u0026db=mysql\u0026ws=nginx\") ただし、MySQLは別でインストールをする必要がある 以下に公式手順をコピペしたものを記載しています\nZabbixリポジトリをインストールする 1 2 3 wget https://repo.zabbix.com/zabbix/7.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_7.0-1+ubuntu24.04_all.deb dpkg -i zabbix-release_7.0-1+ubuntu24.04_all.deb apt update Zabbixサーバー、フロントエンド、エージェントをインストールする 1 apt install zabbix-server-mysql zabbix-frontend-php zabbix-nginx-conf zabbix-sql-scripts zabbix-agent MySQLをインストールする 1 sudo apt install mysql-server 初期データベースを作成する 1 2 mysql -uroot -p password 1 2 3 4 5 mysql\u0026gt; create database zabbix character set utf8mb4 collate utf8mb4_bin; mysql\u0026gt; create user zabbix@localhost identified by \u0026#39;password\u0026#39;; mysql\u0026gt; grant all privileges on zabbix.* to zabbix@localhost; mysql\u0026gt; set global log_bin_trust_function_creators = 1; mysql\u0026gt; quit; Zabbix サーバー ホストで初期スキーマとデータをインポートします。新しく作成したパスワードを入力するよう求められます。 1 zcat /usr/share/zabbix-sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix データベース スキーマをインポートした後、log_bin_trust_function_creators オプションを無効にします。\n1 2 mysql -uroot -p password 1 2 mysql\u0026gt; set global log_bin_trust_function_creators = 0; mysql\u0026gt; quit; Zabbixサーバーのデータベースを構成する ファイル /etc/zabbix/zabbix_server.conf を編集します。\n1 DBPassword=password Zabbixフロントエンド用にPHPを構成する ファイル /etc/zabbix/nginx.conf を編集し、コメントアウトを解除して \u0026rsquo;listen\u0026rsquo; および \u0026lsquo;server_name\u0026rsquo; ディレクティブを設定します。\n1 2 listen 8080; server_name example.com; Zabbixサーバーとエージェントのプロセスを起動する Zabbix サーバーおよびエージェント プロセスを起動し、システムの起動時に起動するようにします。\n1 2 systemctl restart zabbix-server zabbix-agent nginx php8.3-fpm systemctl enable zabbix-server zabbix-agent nginx php8.3-fpm Zabbix UI Webページを開く 1 IPアドレス:8080 初期設定をします\nブラウザの画面 ログインする際の初期ID・パスワードは以下の通りです\nUsername：Admin Password：zabbix 参考URL https://www.zabbix.com/jp/download?zabbix=7.0\u0026os_distribution=ubuntu\u0026os_version=24.04\u0026components=server_frontend_agent\u0026db=mysql\u0026ws=nginx https://www.site24x7.jp/blog/zabbix-6-construction/ 備考：MySQLをアンインストールして再インストールする MySQLをインストールしてZbbixのデータベースを作成する際にエラーになってしまったので、再度インストールをした際の手順です\nアンインストール 1 2 3 4 5 6 sudo apt update sudo apt upgrade sudo apt purge mysql* sudo rm -rf /etx/mysql /var/lib/mysql sudo apt autoremove sudo apt autoclean 再インストール 1 sudo apt install mysql-server ","date":"2024-06-22T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/zabbix-install/","title":"Zbbix7.0LTSをUbuntuにインストールする"},{"content":"環境 Windows 11 Home バージョン 23H2 hugo v0.121.1-extended windows/amd64 hugoのテーマ：stack(\u0026ldquo;https://github.com/CaiJimmy/hugo-theme-stack\") 更新日時の表示 記事のMarkdownファイルに\u0026quot;lastmod:\u0026ldquo;を追加し、更新日時を入れます\n1 2 3 4 title: xxx date: 2024-06-01 lastmod: 2024-06-14 slug: そうすると記事の一番下に更新日時が表示されます\nただ、一番下なので記事を見たとき更新日時をすぐに確認できないので、作成日時の横に更新日時を表示させられるようにします\n作成日時の横に更新日時を表示 下記のパスにある\u0026quot;footer.html\u0026quot;を開きます\n1 ./layouts/partials/article/components/footer.html 上記ファイルの中にある下記の部分をコピーします 下記が更新日時を表示させているコードです\n1 2 3 4 5 6 7 8 {{- if ne .Lastmod .Date -}} \u0026lt;section class=\u0026#34;article-lastmod\u0026#34;\u0026gt; {{ partial \u0026#34;helper/icon\u0026#34; \u0026#34;clock\u0026#34; }} \u0026lt;span\u0026gt; {{ T \u0026#34;article.lastUpdatedOn\u0026#34; }} {{ .Lastmod.Format ( or .Site.Params.dateFormat.lastUpdated \u0026#34;Jan 02, 2006 15:04 MST\u0026#34; ) }} \u0026lt;/span\u0026gt; \u0026lt;/section\u0026gt; {{- end -}} 下記のパスにある\u0026quot;details.html\u0026quot;を開きます\n1 ./layouts/partials/article/components/details.html 下記のコメント（20行目から29行目）を付けた部分に先ほどコピーしたコードを追加します\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 \u0026lt;footer class=\u0026#34;article-time\u0026#34;\u0026gt; {{ if $showDate }} \u0026lt;div\u0026gt; {{ partial \u0026#34;helper/icon\u0026#34; \u0026#34;date\u0026#34; }} \u0026lt;time class=\u0026#34;article-time--published\u0026#34;\u0026gt; {{- .Date.Format (or .Site.Params.dateFormat.published \u0026#34;Jan 02, 2006\u0026#34;) -}} \u0026lt;/time\u0026gt; \u0026lt;/div\u0026gt; {{ end }} {{ if $showReadingTime }} \u0026lt;div\u0026gt; {{ partial \u0026#34;helper/icon\u0026#34; \u0026#34;clock\u0026#34; }} \u0026lt;time class=\u0026#34;article-time--reading\u0026#34;\u0026gt; {{ T \u0026#34;article.readingTime\u0026#34; .ReadingTime }} \u0026lt;/time\u0026gt; \u0026lt;/div\u0026gt; {{ end }} \u0026lt;!--ここから--\u0026gt; {{- if ne .Lastmod .Date -}} \u0026lt;div class=\u0026#34;article-time--lastUpdated\u0026#34;\u0026gt; {{ partial \u0026#34;helper/icon\u0026#34; \u0026#34;clock\u0026#34; }} \u0026lt;time\u0026gt; {{ T \u0026#34;article.lastUpdatedOn\u0026#34; }} {{ .Lastmod.Format ( or .Site.Params.dateFormat.lastUpdated \u0026#34;Jan 02, 2006 15:04 MST\u0026#34; ) }} \u0026lt;/time\u0026gt; \u0026lt;/div\u0026gt; {{- end -}} \u0026lt;!--ここを追加--\u0026gt; \u0026lt;/footer\u0026gt; 確認 下記のコマンドで起動して確認してみます\n1 hugo server -D 無事作成日時の横に更新日時が表示されました\n","date":"2024-06-14T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/hugo-stack-custom-lastupdated/","title":"Hugoで作成日時の横に更新日時を表示できるようにする"},{"content":"ファイル・フォルダ構成 以下のような構成になっています\n1 2 3 4 5 6 dev ┣━ db-data ┣━ log ┣━ mysql ┃　┗━ my.cnf ┗━ docker-compose.yaml docker-compose.yaml のファイル 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 services: mysql: image: mysql:8.4.0 ports: - \u0026#34;3306:3306\u0026#34; environment: MYSQL_ROOT_PASSWORD: mysql MYSQL_DATABASE: db MYSQL_USER: user MYSQL_PASSWORD: password TZ: \u0026#39;Asia/Tokyo\u0026#39; volumes: - ./db-data:/var/lib/mysql - ./mysql:/etc/mysql/conf.d - ./log:/var/log/mysql phpmyadmin: image: phpmyadmin:5.2.1 depends_on: - mysql environment: - PMA_ARBITRARY=1 - PMA_HOSTS=mysql - PMA_USER=root - PMA_PASSWORD=mysql ports: - \u0026#34;3001:80\u0026#34; volumes: db-data: docker composeで構築したMySQLのログをローカルに保存したい my.cnfファイルに以下の内容を追記する\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [mysqld] log_output=FILE # General Log general_log=1 general_log_file=/var/log/mysql/mysql-query.log # Slow Query Log slow_query_log=1 slow_query_log_file=/var/log/mysql/mysql-slow.log # slow_query_time = 1.0s long_query_time=1.0 log_queries_not_using_indexes=0 # Error Log log_error=/var/log/mysql/mysql-error.log log_error_verbosity=3 docker-compose.yamlに以下の内容を追記する\n1 2 volumes: - ./log:/var/log/mysql ログが書き込まれない パーミッションがありすぎるとログファイルが生成されないので権限を必要最小限にする\n1 sudo chmod -R 775 . 1 2 3 4 5 mao@mao:~/dev$ sudo ls -l ./log total 32 -rw-r----- 1 999 systemd-journal 16239 6月 1 23:33 mysql-error.log -rw-r----- 1 999 systemd-journal 10468 6月 1 23:29 mysql-query.log -rw-r----- 1 999 systemd-journal 180 6月 1 23:29 mysql-slow.log docker-composeのスタートとストップ スタート\n1 sudo docker compose up -d ストップ\n1 sudo docker compose down -v 以下のURLからphpmyadminにアクセスし、少し作業をします\nするとログファイルを作成されます\nhttp://localhost:3001/ ","date":"2024-06-02T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/docker-compose-mysql-log/","title":"docker-composeで構築したMySQLのログをローカルに保存する"},{"content":"発生現象 Deskmini x300(CPU:Ryzen7 5700G) に元々SATA SSDを差していたがデータをM.2 SSDにクローンをして差し替えると、有線LANが繋がらなくなる\n結果、ProxmoxのWebGUIにアクセスできなくなる\n対処方法 WebGUIではなく本体からコマンドで、ブリッジネットワークにリンクしている物理LANを「enp1s0」から「enp2s0」へ変更する\n手順 SATA SSDのとき 元々は500GBのSATA SSDを使用していた ネットワークの設定 M.2 SSDのとき M.2 SSDに差し替えた 元々「enp1s0」だったが「enp2s0」にすると通信可能になった ","date":"2024-05-26T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/proxmox-ssd-lan/","title":"Deskmini x300でSATA SSDからM.2 SSDに差し替えるとProxmoxのWebGUIにアクセスできなくなる"},{"content":"環境 Ubuntu 23.10 Intel Core i5 13500 メモリ64GB 1：setup ソースからビルドするのに必要なソフトを確認する\n1 2 make --version gcc --version インストールされていない場合は以下のコマンドでインストールする\n1 2 sudo apt install make sudo apt install gcc ビルドするのに必要なバージョン\nThe minimum version of Go required depends on the target version of Go:\nGo \u0026lt;= 1.4: a C toolchain. 1.5 \u0026lt;= Go \u0026lt;= 1.19: a Go 1.4 compiler. 1.20 \u0026lt;= Go \u0026lt;= 1.21: a Go 1.17 compiler. 1.22 \u0026lt;= Go \u0026lt;= 1.23: a Go 1.20 compiler. Going forward, Go version 1.N will require a Go 1.M compiler, where M is N-2 rounded down to an even number. Example: Go 1.24 and 1.25 require Go 1.22. 2：build go1.4 go1.4-bootstrapをビルドする\n1 2 3 wget https://dl.google.com/go/go1.4-bootstrap-20171003.tar.gz mkdir go1.4-bootstrap \u0026amp;\u0026amp; tar xzvf go1.4-bootstrap-20171003.tar.gz -C go1.4-bootstrap --strip-components 1 cd ./go1.4-bootstrap/src 1 CGO_ENABLED=0 bash ./make.bash 1 2 Installed Go for linux/amd64 in /home/mao/Desktop/go1.4-bootstrap Installed commands in /home/mao/Desktop/go1.4-bootstrap/bin 3：build go1.17 go1.17をビルドする\n1 2 3 wget https://dl.google.com/go/go1.17.src.tar.gz mkdir go1.17 \u0026amp;\u0026amp; tar xzvf go1.17.src.tar.gz -C go1.17 --strip-components 1 cd ./go1.17/src 1 2 GOROOT_BOOTSTRAP=${PWD}/go1.4-bootstrap bash ./all.bash GOROOT_BOOTSTRAP=/home/mao/Desktop/go1.4-bootstrap bash ./all.bash 1 2 3 4 5 6 7 Go version is \u0026#34;go1.17\u0026#34;, ignoring -next /home/mao/Desktop/go1.17/api/next.txt ALL TESTS PASSED --- Installed Go for linux/amd64 in /home/mao/Desktop/go1.17 Installed commands in /home/mao/Desktop/go1.17/bin *** You need to add /home/mao/Desktop/go1.17/bin to your PATH. 4：build go1.20 go1.20をビルドする\n1 2 3 wget https://dl.google.com/go/go1.20.src.tar.gz mkdir go1.20 \u0026amp;\u0026amp; tar xzvf go1.20.src.tar.gz -C go1.20 --strip-components 1 cd ./go1.20/src 1 2 /home/mao/Desktop/go1.17/bin GOROOT_BOOTSTRAP=/home/mao/Desktop/go1.17 bash ./all.bash 1 2 3 4 5 ALL TESTS PASSED --- Installed Go for linux/amd64 in /home/mao/Desktop/go1.20 Installed commands in /home/mao/Desktop/go1.20/bin *** You need to add /home/mao/Desktop/go1.20/bin to your PATH. 5：build go1.22.2 latest go1.22.2をビルドする\n1 2 3 wget https://dl.google.com/go/go1.22.2.src.tar.gz mkdir go1.22.2 \u0026amp;\u0026amp; tar xzvf go1.22.2.src.tar.gz -C go1.22.2 --strip-components 1 cd ./go1.22.2/src 1 2 /home/mao/Desktop/go1.20/bin GOROOT_BOOTSTRAP=/home/mao/Desktop/go1.20 bash ./all.bash 1 2 3 4 5 ALL TESTS PASSED --- Installed Go for linux/amd64 in /home/mao/Desktop/go1.22.2 Installed commands in /home/mao/Desktop/go1.22.2/bin *** You need to add /home/mao/Desktop/go1.22.2/bin to your PATH. 6：Pathを通す パスを通してバージョンを確認する\n1 sudo cp -rp ./go1.22.2 /usr/local/ .bashrc\n1 2 export PATH=$PATH:/usr/local/go/bin export PATH=$PATH:/usr/local/go1.22.2/bin 1 source ~/.bashrc 1 go version 1 go version go1.22.2 linux/amd64 参考URL https://go.dev/doc/install/source https://qiita.com/myoshimi/items/5d1f6a2ee8a849bac7eb https://qiita.com/soarflat/items/d5015bec37f8a8254380 ","date":"2024-05-25T00:00:00Z","permalink":"https://tiisanamaou.github.io/post/golang-source-build/","title":"Go言語をソースコードからビルドする"}]