I0714 05:20:57.825802 1 cephcsi.go:191] Driver version: v3.11.0 and Git version: bc24b5eca87626d690a29effa9d7420cc0154a7a I0714 05:20:57.825929 1 cephcsi.go:223] Starting driver type: controller with name: rbd.csi.ceph.com I0714 05:20:57.826774 1 leaderelection.go:250] attempting to acquire leader lease kube-system/rbd.csi.ceph.com-kube-system... I0714 05:20:57.834471 1 leaderelection.go:354] lock is held by ceph-csi-rbd-provisioner-69cb97fb6-gf7g2_d3db0bb9-71aa-475a-bfb5-4c63a245d806 and has not yet expired I0714 05:20:57.834487 1 leaderelection.go:255] failed to acquire lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:02.237401 1 leaderelection.go:354] lock is held by ceph-csi-rbd-provisioner-69cb97fb6-gf7g2_d3db0bb9-71aa-475a-bfb5-4c63a245d806 and has not yet expired I0714 05:21:02.237438 1 leaderelection.go:255] failed to acquire lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:05.329484 1 leaderelection.go:354] lock is held by ceph-csi-rbd-provisioner-69cb97fb6-gf7g2_d3db0bb9-71aa-475a-bfb5-4c63a245d806 and has not yet expired I0714 05:21:05.329524 1 leaderelection.go:255] failed to acquire lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:07.822612 1 leaderelection.go:354] lock is held by ceph-csi-rbd-provisioner-69cb97fb6-gf7g2_d3db0bb9-71aa-475a-bfb5-4c63a245d806 and has not yet expired I0714 05:21:07.822659 1 leaderelection.go:255] failed to acquire lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:10.729667 1 leaderelection.go:354] lock is held by ceph-csi-rbd-provisioner-69cb97fb6-gf7g2_d3db0bb9-71aa-475a-bfb5-4c63a245d806 and has not yet expired I0714 05:21:10.729691 1 leaderelection.go:255] failed to acquire lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:13.088429 1 leaderelection.go:260] successfully acquired lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:13.091964 1 reflector.go:289] Starting reflector *v1.PersistentVolume (10h9m9.705937983s) from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0714 05:21:13.091984 1 reflector.go:325] Listing and watching *v1.PersistentVolume from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0714 05:21:13.093504 1 reflector.go:351] Caches populated for *v1.PersistentVolume from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0714 05:21:13.094532 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:15.101099 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:17.113417 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:19.122144 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:21.132617 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:23.141953 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:25.150423 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:27.159363 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:29.170218 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:31.180758 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:33.189795 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:35.199151 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:37.208063 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:39.218525 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:41.229316 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:43.238455 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:45.248436 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:47.257517 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:49.264676 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:51.274718 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:53.283334 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:55.292064 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:57.300859 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:21:59.310882 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:01.318330 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:03.331513 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:05.338110 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:07.348254 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:09.356224 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:11.365448 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:13.376004 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:16.479826 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:18.685425 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:20.694693 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:22.707474 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:24.715889 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:26.727147 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:28.736438 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:30.743020 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:32.753707 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:34.764104 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:36.774400 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:38.781686 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:40.791650 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:42.816151 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:44.824125 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:53.368738 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:55.380577 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:57.389709 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:22:59.396428 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:01.406585 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:03.414011 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:05.422805 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:07.431326 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:09.437900 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:11.446430 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system [controller-runtime] log.SetLogger(...) was never called; logs will not be displayed. Detected at: > goroutine 89 [running]: > runtime/debug.Stack() > /usr/local/go/src/runtime/debug/stack.go:24 +0x5e > sigs.k8s.io/controller-runtime/pkg/log.eventuallyFulfillRoot() > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/log/log.go:60 +0xcd > sigs.k8s.io/controller-runtime/pkg/log.(*delegatingLogSink).WithValues(0xc000826a40, {0xc000853620, 0x6, 0x6}) > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/log/deleg.go:168 +0x49 > github.com/go-logr/logr.Logger.WithValues(...) > /go/src/github.com/ceph/ceph-csi/vendor/github.com/go-logr/logr/logr.go:332 > sigs.k8s.io/controller-runtime/pkg/controller.NewUnmanaged.func1(0xc000984b20) > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/controller/controller.go:121 +0x1d7 > sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0000d4640, {0x27c0488, 0xc0008289b0}, {0x1f93660?, 0xc000984b00?}) > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:306 +0x16a > sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0000d4640, {0x27c0488, 0xc0008289b0}) > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266 +0x1af > sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2() > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227 +0x79 > created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 76 > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:223 +0x565 I0714 05:23:11.682581 1 reflector.go:289] Starting reflector *v1.Secret (10h55m51.667707246s) from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0714 05:23:11.682622 1 reflector.go:325] Listing and watching *v1.Secret from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0714 05:23:11.728601 1 reflector.go:351] Caches populated for *v1.Secret from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0714 05:23:11.810206 1 omap.go:89] got omap values: (pool="kube", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-ca1874cb-c781-414c-931c-0b4dd6158f08:60e69879-4cca-4216-a2ab-b42759f6c23a] I0714 05:23:11.811076 1 omap.go:89] got omap values: (pool="kube", namespace="", name="csi.volume.60e69879-4cca-4216-a2ab-b42759f6c23a"): map[csi.imageid:37f3ac2142c0 csi.imagename:csi-vol-60e69879-4cca-4216-a2ab-b42759f6c23a csi.volname:pvc-ca1874cb-c781-414c-931c-0b4dd6158f08 csi.volume.owner:default] I0714 05:23:11.883029 1 omap.go:89] got omap values: (pool="kube", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-ca1874cb-c781-414c-931c-0b4dd6158f08:60e69879-4cca-4216-a2ab-b42759f6c23a] I0714 05:23:11.884118 1 omap.go:89] got omap values: (pool="kube", namespace="", name="csi.volume.60e69879-4cca-4216-a2ab-b42759f6c23a"): map[csi.imageid:37f3ac2142c0 csi.imagename:csi-vol-60e69879-4cca-4216-a2ab-b42759f6c23a csi.volname:pvc-ca1874cb-c781-414c-931c-0b4dd6158f08 csi.volume.owner:default] I0714 05:23:13.455104 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:15.471274 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:17.482323 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:19.493337 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:21.504253 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:23.515815 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:25.525653 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:27.533289 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:29.541729 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:31.551543 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:33.561182 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:35.572965 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:37.582429 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:39.591968 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:41.609495 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:43.617281 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:45.628259 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:47.637458 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:49.646196 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:51.654665 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:53.661952 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:55.672657 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:57.682786 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:23:59.690668 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:01.701529 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:03.710699 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:05.719997 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:07.729869 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:09.739923 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:11.748482 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:13.758471 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:15.768535 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:17.776780 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:19.786098 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:21.799815 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:23.811171 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:25.819415 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:27.826644 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:29.833804 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:31.841923 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:33.851049 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:35.857500 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:37.865458 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:39.873034 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:41.909945 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:43.961390 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:45.966873 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:47.974546 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:50.056307 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:52.154604 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:54.195246 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:56.438442 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:24:58.477155 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0714 05:25:00.510419 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system