Skip to content

High disk utilization on large installations caused by this plugin #179

@krf

Description

@krf

What feature do you want to see added?

I just went down the rabbit hole figuring out why our Jenkins UI was slow at times, which then might us notice regular high disk utilization on our Grafana monitoring instance.

This is the Grafana reporting on our Linux VM hosting the Jenkins installation:

Image

As you can see after a quiet period of 15 mins there's a highly active period of around 25 mins where there's lots of disk I/O (mostly reads).

That lead me to inspecting the Jenkins' JVM threads where I stumbled upon:

"Simple disk usage computation [#1]" Id=332 Group=main RUNNABLE
  at java.base@17.0.17/sun.nio.fs.UnixNativeDispatcher.readdir(Native Method)
  at java.base@17.0.17/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.readNextEntry(Unknown Source)
  at java.base@17.0.17/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.hasNext(Unknown Source)
  -  locked sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator@79673db3
  at java.base@17.0.17/java.nio.file.FileTreeWalker.next(Unknown Source)
  at java.base@17.0.17/java.nio.file.Files.walkFileTree(Unknown Source)  

and then https://github.com/jenkinsci/cloudbees-disk-usage-simple-plugin/blob/master/src/main/java/com/cloudbees/simplediskusage/QuickDiskUsagePlugin.java#L59 (which in fact is that 15 mins quiet period I was experiencing in the monitoring).

Arguably our installation is very large (Jenkins' home is multiple 100Gs), but still this plugin should probably take some precautions to not cause such high disk I/O.

Some random idea to mitigate this at least:
Maybe add some warning popup for admins in Jenkins whenever that scan jobs takes more than, say, 5 minutes? So you'd at least notice that something (bad) is going on. It's difficult to even notice that without the proper monitoring.

Upstream changes

No response

Are you interested in contributing this feature?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No fields configured for Enhancement.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions