Whitelisting or blacklisting metrics

[zefir6]

It would be very useful to be able to whitelist or blacklist specific metrics. In some situations you have to use managed prometheus, and may not be able to filter specific metrics on prometheus level, and exporter generates a lot of entries especially for metrics that have user_name tag.

[marcelmay]

@zefir6 , you could try to reduce the buckets fsimage_user_fsize_bucket buckets for a start , or completely disable file size distribution for users.

Reduce the number of buckets:

# Configure file size distribution buckets, supporting IEC units of KiB, MiB, GiB, TiB, PiB
fileSizeDistributionBuckets: ['0','1MiB', '32MiB', '128MiB', '1GiB']

Disable any file size distribution on user level:

skipFileDistributionForUserStats : true

The file size distribution on user level is quite useful for catching culprits that create too many small files on HDFS though...

[zefir6]

Hey, thanks for reply :)
Regardless of how I set that option, I still get hundreds of lines like this:

fsimage_group_blocks{group_name="group.name",} 0.0
fsimage_user_blocks{user_name="user.name",} 0.0

(and the same for many other attributes)

According to my count, with every skip enabled, every user (even if he has one file), generates 15 lines in the exporter. I was just considering editing the code and trying to cut out the bit that generates user & group labels (as thats about peak of my java skills :) )

[marcelmay]

Can you provide me the output of number of metrics?

sort_desc(count({__name__=~"fsimage_.*"}) by (__name__))

Usually the mentioned before histogram metrics for file size distribution have a real impact (number of users X number of buckets ...).

[marcelmay]

... and the config if possible