[Bug]: JVM fatal crash on fresh deploy — $LOGDIR not created on PVC-backed mount

[webdz9r]

Is this a critical security issue?

• This is not a security issue.

Describe the Bug

When deploying the openvoxdb container with a PersistentVolumeClaim (PVC) mounted at
/opt/puppetlabs/server/data/puppetdb/, the JVM fails to start on every fresh deployment.

The Containerfile creates $LOGDIR at build time:

mkdir -p "$LOGDIR" && \
chown puppetdb:puppetdb "$LOGDIR" && \

And OPENVOXDB_JAVA_ARGS references it for GC logging:

-Xlog:gc*:file=$LOGDIR/puppetdb_gc.log

However, when a PVC is mounted at the parent path in Kubernetes, the mount overlays the entire directory with an empty volume — erasing the logs/ subdirectory created at build time. The container's entrypoint scripts do not recreate it before starting the JVM, causing a fatal startup error.

This affects all tested image tags: 8.9.0-main, 8.11.0-main, and 8.12.1-latest.

Workaround: Use the Helm chart's puppetdb.extraInitContainers:

puppetdb:
  extraInitContainers:
    - name: create-log-dir
      image: busybox:1.37
      command: ["sh", "-c", "mkdir -p /data/logs && chown 999:999 /data/logs"]
      volumeMounts:
        - name: puppetdb-storage
          mountPath: /data

Expected Behavior

The container should ensure $LOGDIR exists during entrypoint initialization, before the JVM is launched. A one-line mkdir -p "$LOGDIR" in the entrypoint scripts would suffice.

Steps to Reproduce

1. Deploy the openvoxdb container in Kubernetes with a PVC mounted at /opt/puppetlabs/server/data/puppetdb/ (e.g., via the openvox/puppetserver Helm chart v10.0.1)
2. Ensure the PVC is empty (fresh deployment, no pre-existing data)
3. Set OPENVOXDB_POSTGRES_HOSTNAME to a reachable PostgreSQL instance
4. Wait for init containers to pass (pgchecker, wait-puppetserver)
5. Observe the puppetdb container crash with a fatal JVM error

Environment

• OpenVoxDB container image: ghcr.io/openvoxproject/openvoxdb:8.12.1-latest (also reproduced with 8.11.0-main and 8.9.0-main)
• OpenVox Server: ghcr.io/openvoxproject/openvoxserver:8.12.1-main
• Helm chart: openvox/puppetserver v10.0.1
• Kubernetes: GKE v1.33.5-gke.2326000
• Storage: GKE standard-rwo (pd.csi.storage.gke.io), ReadWriteOnce PVCs
• PostgreSQL: Cloud SQL PostgreSQL 16 (external, connected via private IP)

Additional Context

This is a Kubernetes-specific issue. In Docker Compose deployments, named volumes don't overlay the entire directory tree, so the logs/ directory created at build time persists. In Kubernetes with a PVC mount, the entire parent directory is replaced with an empty volume.

Originally filed as OpenVoxProject/openvox-server#215 (wrong repo, now closed).

Relevant log output

Running /container-entrypoint.d/30-certificate-allowlist.sh
[0.001s][error][logging] Error opening log file '/opt/puppetlabs/server/data/puppetdb/logs/puppetdb_gc.log': No such file or directory
[0.001s][error][logging] Initialization of output 'file=/opt/puppetlabs/server/data/puppetdb/logs/puppetdb_gc.log' using options '(null)' failed.
Invalid -Xlog option '-Xlog:gc*:file=/opt/puppetlabs/server/data/puppetdb/logs/puppetdb_gc.log', see error log for details.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Fix submitted

PR: #68