How to deal a large number of small files in GraphAr ? #630
DancyHuang
started this conversation in
Ideas
Replies: 1 comment
-
We have a goal to create a CLI-tool for working with GraphAr data. It is on roadmap. I think, that |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
For some large scale graphs , when use GraphAr as the storage format. if the underlying storage is HDFS or another object storage system, a large number of small files can have a significant impact on the file system, such as placing considerable pressure on metadata in the case of HDFS.
Has the Graphr community considered consolidating a large number of small files?
Beta Was this translation helpful? Give feedback.
All reactions