Healthcheck failing with panic 'unknown userid 1003' since 10.10.1 update, but Mattermost runs fine

[rumpelbaer]

Description

After updating to Mattermost Team Edition 10.10.1 (Docker image mattermost/mattermost-team-edition:latest), the container status is now always reported as unhealthy, although Mattermost itself is running normally and is fully accessible.

The docker inspect output for the healthcheck shows repeated panic: user: unknown userid 1003 errors. This was not happening before the update.

---

Healthcheck log excerpt

panic: user: unknown userid 1003

goroutine 1 [running]:
github.com/mattermost/mattermost/server/v8/cmd/mmctl/commands.init.1()
    github.com/mattermost/mattermost/server/v8/cmd/mmctl/commands/auth_utils.go:49 +0x4e

---

Full healthcheck output example from docker inspect

"Health": {
    "Status": "unhealthy",
    "FailingStreak": 28592,
    "Log": [
        {
            "Start": "2025-08-11T09:36:30.619061811+02:00",
            "End": "2025-08-11T09:36:30.708679359+02:00",
            "ExitCode": 2,
            "Output": "panic: user: unknown userid 1003\n\ngoroutine 1 [running]:\ngithub.com/mattermost/mattermost/server/v8/cmd/mmctl/commands.init.1()\n\tgithub.com/mattermost/mattermost/server/v8/cmd/mmctl/commands/auth_utils.go:49 +0x4e\n"
        }
    ]
}

---

Environment

• Docker Compose setup with mattermost and postgres:16 as DB
• Mattermost image: mattermost/mattermost-team-edition:latest
• Host OS: SLES 15 SP6
• Running as a custom user (user: "1003:471")
• Note: UID 1003 is the local user on the host system that owns and has access to the /opt/docker/ directories used as volumes for the containers.
• Healthcheck failing continuously since updating to 10.10.1
• Mattermost web UI and API work normally despite the unhealthy status

---

docker-compose.yml snippet

services:
  mattermost-db:
    image: postgres:16
    container_name: mattermost-db
    hostname: mattermost-db
    restart: always
    environment:
      POSTGRES_DB: mattermost
      POSTGRES_USER: mattermostuser
      POSTGRES_PASSWORD: mattermostpw
    volumes:
      - /opt/docker/mattermost/db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-d", "mattermost", "-U", "mattermostuser"]
      interval: 30s
      timeout: 20s
      retries: 3
    networks:
      mattermost:
        ipv4_address: 172.x.x.x

  mattermost:
    image: mattermost/mattermost-team-edition:latest
    container_name: mattermost
    restart: unless-stopped
    depends_on:
      - mattermost-db    
    user: "1003:471"
    environment:
      MM_SQLSETTINGS_DRIVERNAME: postgres
      MM_SQLSETTINGS_DATASOURCE: postgres://... 
      MM_BLEVESETTINGS_INDEXDIR: /mattermost/bleve-indexes
      MM_SERVICESETTINGS_SITEURL: https://chat.domain.tld
      TZ: Europe/Berlin
    volumes:
      - /opt/docker/mattermost/config:/mattermost/config
      - /opt/docker/mattermost/data:/mattermost/data
      - /opt/docker/mattermost/logs:/mattermost/logs
      - /opt/docker/mattermost/plugins:/mattermost/plugins
      - /opt/docker/mattermost/client/plugins:/mattermost/client/plugins
      - /opt/docker/mattermost/indexes:/mattermost/bleve-indexes
    networks:
      mattermost:
        ipv4_address: 172.x.x.x
      proxy:
        ipv4_address: 172.x.x.x

networks:
  proxy:
    external: true

  mattermost:
    internal: true
    name: mattermost
    ipam:
      driver: default
      config:
        - subnet: 172.x.x.x/16

---

Possible cause

It seems the healthcheck command (likely running mmctl) is executed in the container as a user that does not exist in /etc/passwd when a custom user: is set in Docker Compose.
This causes the panic: user: unknown userid error, leading to a failed healthcheck even though the server itself is fine.

---

Questions

1. Is this an intended change in 10.10.1?
2. Should the healthcheck be adjusted to work when the container runs as a non-root custom UID/GID?
3. Or should the image add a passwd entry for the current UID to avoid the panic?

[arnabJ]

I am facing a similar issue with 10.11.2 (Updating from 10.8.1). But for me it gives 404 not found when it's unhelathy. That could be due to how my Traefik is setup.

Error:

panic: user: unknown userid 1000 goroutine 1 [running]:
github.com/mattermost/mattermost/server/v8/cmd/mmctl/commands.init.1()
github.com/mattermost/mattermost/server/v8/cmd/mmctl/commands/auth_utils.go:49 +0x4e

My compose file:

services:
    mattermost:
        image: mattermost/mattermost-enterprise-edition:${MATTERMOST_IMAGE_TAG}
        container_name: mattermost
        restart: unless-stopped
        security_opt:
            - no-new-privileges:true
        pids_limit: 200
        read_only: false
        user: "1000:1000"
        tmpfs:
            - /tmp
        ports:
            # - ${APP_PORT}:8065
            - ${CALLS_PORT}:${CALLS_PORT}/udp
            - ${CALLS_PORT}:${CALLS_PORT}/tcp
        volumes:
            - ${MATTERMOST_CONFIG_PATH}:/mattermost/config:rw
            - ${MATTERMOST_DATA_PATH}:/mattermost/data:rw
            - ${MATTERMOST_LOGS_PATH}:/mattermost/logs:rw
            - ${MATTERMOST_PLUGINS_PATH}:/mattermost/plugins:rw
            - ${MATTERMOST_CLIENT_PLUGINS_PATH}:/mattermost/client/plugins:rw
            - ${MATTERMOST_BLEVE_INDEXES_PATH}:/mattermost/bleve-indexes:rw
        networks:
            - traefik
        environment:
            - TZ
            - MM_SQLSETTINGS_DRIVERNAME
            - MM_SQLSETTINGS_DATASOURCE
            - MM_BLEVESETTINGS_INDEXDIR
            - MM_SERVICESETTINGS_SITEURL
            - MM_CALLS_GROUP_CALLS_ALLOWED
        labels:
            - "traefik.enable=true"
            - "traefik.http.routers.mattermost.rule=Host(`mattermost.example.com`)"
            - "traefik.http.routers.mattermost.entrypoints=websecure"
            - "traefik.http.routers.mattermost.tls.certresolver=letsencrypt"

networks:
    traefik:
        external: true

[smuskiew]

Seems like nothing with the mmctl command works when the container runs as a custom user, even on earlier versions of Mattermost (I've got 10.9.5 running). Any mmctl command I attempt gives the panic: user: unknown userid <id> error.

The problem with 10.10 onward is now the healthcheck is using mmctl by default so it prevents the container from ever reaching "healthy" status (which for me means my Traefik instance never recognizes MM as up and I can no longer reach it)

Are there any solutions/workarounds for getting mmctl to work when the container runs under a different userid?

[StephanRoemer]

Facing the same issue. A solution would be much appreciated!
Even more since Mattermost informs the user that running as root is not recommended.

[arnabJ]

@StephanRoemer @smuskiew @rumpelbaer

I made it work by manually disabling the Health Check. This is not an ideal solution but at least it works as a workaround.

I have mentioned the same here: mattermost/mattermost#36224 (comment)