I0918 02:27:34.685903 1 cephcsi.go:171] Driver version: v3.5.1 and Git version: 00f88e58a0b08be3d7ea009a0cfde8dca23021bf I0918 02:27:34.686033 1 cephcsi.go:220] Starting driver type: controller with name: rbd.csi.ceph.com I0918 02:27:34.787267 1 request.go:597] Waited for 89.087221ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1beta3?timeout=32s I0918 02:27:34.792925 1 shared_informer.go:270] caches populated I0918 02:27:34.793089 1 leaderelection.go:248] attempting to acquire leader lease kube-system/rbd.csi.ceph.com-kube-system... I0918 02:27:34.814387 1 leaderelection.go:258] successfully acquired lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:34.815269 1 reflector.go:219] Starting reflector *v1.PersistentVolume (9h37m26.225955903s) from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:250 I0918 02:27:34.815286 1 reflector.go:255] Listing and watching *v1.PersistentVolume from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:250 I0918 02:27:34.831310 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:34.915266 1 shared_informer.go:270] caches populated I0918 02:27:34.915342 1 shared_informer.go:270] caches populated I0918 02:27:36.858221 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:38.878265 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:40.898231 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:42.920799 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:44.961900 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:46.982482 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:49.004282 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:51.023196 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:53.044329 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:55.065643 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:57.086219 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:27:59.108983 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:01.149884 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:03.174101 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:05.194821 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:07.211478 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:09.238979 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:11.255815 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:13.276227 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:15.306597 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:17.327345 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:19.348522 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:21.366789 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:23.390014 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:25.422336 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:27.451171 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:29.472278 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:31.494186 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:33.519953 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:35.544209 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:37.568081 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:39.590262 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:41.611696 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:43.637628 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:45.658202 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:47.678492 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:49.695548 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:51.718205 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:53.751175 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:55.769338 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:57.795919 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:28:59.817702 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:01.839065 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:03.858695 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:05.880217 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:07.902053 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:09.924950 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:11.946279 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:13.968488 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:15.993153 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:18.035052 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:20.056486 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:22.075116 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:24.098005 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:26.114836 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:28.139924 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:30.160664 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:32.181315 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:34.206762 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:36.229297 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:38.254335 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:40.276544 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:42.301092 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:44.318558 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:46.339935 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:48.356825 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:50.376902 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:52.399011 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:54.426558 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:56.447118 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:29:58.466808 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:00.487080 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:02.503936 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:04.519536 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:06.544697 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:08.579050 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:10.602884 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:12.624739 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:14.647157 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:16.665988 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:18.678208 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:20.711793 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:22.731339 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:24.750763 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:26.770262 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:28.803842 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:30.823090 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:32.838255 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:34.860809 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:36.900721 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:38.916672 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:40.963256 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:42.982499 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:45.005677 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:47.023575 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:49.043886 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:51.056008 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:53.076944 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:55.099459 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:57.122561 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:30:59.142844 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:01.163268 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:03.186118 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:05.209223 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:07.231956 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:09.252198 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:11.283543 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:13.302350 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:15.322958 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:17.339892 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:19.359489 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:21.425255 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:23.495280 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:25.524818 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:27.590895 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:29.613406 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:31.638139 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:33.717544 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:35.739172 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:37.763689 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:39.787091 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:41.810380 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:43.844158 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:45.865528 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:47.888986 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:49.904768 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:51.927990 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:53.947310 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:55.968804 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:31:57.985968 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:00.002261 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:01.805534 1 reflector.go:219] Starting reflector *v1.Secret (9h40m51.923496738s) from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:250 I0918 02:32:01.805667 1 reflector.go:255] Listing and watching *v1.Secret from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:250 I0918 02:32:01.901099 1 shared_informer.go:270] caches populated I0918 02:32:01.918432 1 omap.go:87] got omap values: (pool="kube", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-29d7d5c3-cd8a-4d53-ab0e-d2441fda8810:212048ce-b309-11f1-a98c-1ee9ce2dde43] I0918 02:32:01.920372 1 omap.go:87] got omap values: (pool="kube", namespace="", name="csi.volume.212048ce-b309-11f1-a98c-1ee9ce2dde43"): map[csi.imageid:37f4ee95a779 csi.imagename:csi-vol-212048ce-b309-11f1-a98c-1ee9ce2dde43 csi.volname:pvc-29d7d5c3-cd8a-4d53-ab0e-d2441fda8810 csi.volume.owner:default] I0918 02:32:01.921650 1 omap.go:87] got omap values: (pool="kube", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-29d7d5c3-cd8a-4d53-ab0e-d2441fda8810:212048ce-b309-11f1-a98c-1ee9ce2dde43] I0918 02:32:01.922506 1 omap.go:87] got omap values: (pool="kube", namespace="", name="csi.volume.212048ce-b309-11f1-a98c-1ee9ce2dde43"): map[csi.imageid:37f4ee95a779 csi.imagename:csi-vol-212048ce-b309-11f1-a98c-1ee9ce2dde43 csi.volname:pvc-29d7d5c3-cd8a-4d53-ab0e-d2441fda8810 csi.volume.owner:default] I0918 02:32:02.022447 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:04.042848 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:06.055832 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:08.075766 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:10.094670 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:12.118280 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:14.136207 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:16.156397 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:18.178571 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:20.198258 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:22.216088 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:24.242294 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:26.259157 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:28.276898 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:30.289607 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:32.312252 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:34.324877 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:36.339425 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:38.358222 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:40.375335 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:42.393167 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:44.411736 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:46.431123 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0918 02:32:48.448440 1 leaderelection.go:278] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system