Skip to content

Chalk memory usage #2945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
matklad opened this issue Jan 29, 2020 · 6 comments
Closed

Chalk memory usage #2945

matklad opened this issue Jan 29, 2020 · 6 comments
Labels
E-hard E-unknown It's unclear if the issue is E-hard or E-easy without digging in fun A technically challenging issue with high impact

Comments

@matklad
Copy link
Member

matklad commented Jan 29, 2020

running analysis-stats --memory-usage on the substrate repo I get the following result:

Inference: 149.225896663s, 12318mb allocated 12693mb resident
Total: 176.298093521s, 12318mb allocated 12693mb resident
  6790mb TraitSolverQuery
  1738mb TraitSolveQuery (deps)
   806mb ImplDatumQuery (deps)
   536mb ImplDatumQuery
   457mb MacroArgQuery
   293mb ComputeCrateDefMapQuery
   123mb ImplsForTraitQuery (deps)
    81mb DoInferQuery (deps)
    76mb RawItemsQuery
    61mb BodyQuery
    60mb AssociatedTyValueQuery (deps)
    50mb DoInferQuery
    42mb ImplTraitQuery (deps)
    41mb ImplSelfTyQuery (deps)
    38mb ImplDataQuery (deps)
    38mb ImplDataQuery
    38mb ImplsForTraitQuery
    34mb GenericParamsQuery (deps)
    26mb GenericParamsQuery
    26mb BodyWithSourceMapQuery
    26mb AstIdMapQuery (deps)
    25mb AssociatedTyValueQuery
    25mb MacroArgQuery (deps)
    24mb ParseMacroQuery (deps)
    24mb MacroExpandQuery (deps)
    22mb MacroDefQuery
    20mb GenericPredicatesQuery (deps)
    18mb RawItemsQuery (deps)
    18mb ComputeCrateDefMapQuery (deps)
    14mb ParseQuery
    13mb AstIdMapQuery
     9mb TyQuery (deps)
     8mb ImplTraitQuery
     8mb BodyWithSourceMapQuery (deps)
     7mb InternFunctionQuery
     7mb FunctionDataQuery
     7mb GenericDefaultsQuery (deps)
     6mb ExprScopesQuery
     5mb AttrsQuery (deps)
     5mb GenericPredicatesQuery
     4mb ImplsInCrateQuery (deps)
     4mb AttrsQuery
  4080kb InternConstQuery
  3380kb FunctionDataQuery (deps)
  3312kb CallableItemSignatureQuery (deps)
  2970kb TypeAliasDataQuery
  2880kb BodyQuery (deps)
  2877kb TraitDatumQuery
  2797kb TypeAliasDataQuery (deps)
  2235kb TyQuery
  2118kb ExprScopesQuery (deps)
  1924kb StructDatumQuery (deps)
  1916kb ParseQuery (deps)
  1565kb EnumDataQuery
  1546kb CallableItemSignatureQuery
  1394kb StructDataQuery
  1390kb TraitDatumQuery (deps)
  1379kb ValueTyQuery (deps)
  1113kb ImplsInCrateQuery
  1092kb GenericPredicatesForParamQuery (deps)
   985kb InternTypeAliasQuery
   876kb GenericDefaultsQuery
   808kb FieldTypesQuery (deps)
   746kb StructDatumQuery
   721kb ParseMacroQuery
   670kb TraitSolveQuery
   656kb ModuleLangItemsQuery (deps)
   578kb GenericPredicatesForParamQuery
   548kb ImplSelfTyQuery
   539kb MacroDefQuery (deps)
   457kb StructDataQuery (deps)
   345kb FieldTypesQuery
   341kb TraitDataQuery (deps)
   294kb ValueTyQuery
   260kb LangItemQuery (deps)
   173kb TraitDataQuery
   157kb EnumDataQuery (deps)
   136kb InternImplQuery
   133kb AssociatedTyDataQuery (deps)
    86kb ConstDataQuery (deps)
    71kb MacroExpandQuery
    64kb InternStructQuery
    57kb AssociatedTyDataQuery
    44kb InternStaticQuery
    43kb ConstDataQuery
    36kb CrateLangItemsQuery (deps)
    36kb InternEnumQuery
     8kb CrateLangItemsQuery
   4096b StaticDataQuery (deps)
   4096b ModuleLangItemsQuery
   4096b InternTraitQuery
      0b Synthetic Write
      0b SourceRootCratesQuery
      0b SourceRootCratesQuery (deps)
      0b InternMacroQuery
      0b InternMacroQuery (deps)
      0b UnionDataQuery
      0b UnionDataQuery (deps)
      0b StaticDataQuery
      0b LangItemQuery
      0b DocumentationQuery
      0b DocumentationQuery (deps)
      0b InternFunctionQuery (deps)
      0b InternStructQuery (deps)
      0b InternUnionQuery
      0b InternUnionQuery (deps)
      0b InternEnumQuery (deps)
      0b InternConstQuery (deps)
      0b InternStaticQuery (deps)
      0b InternTraitQuery (deps)
      0b InternTypeAliasQuery (deps)
      0b InternImplQuery (deps)
      0b TraitSolverQuery (deps)
      0b InternTypeCtorQuery
      0b InternTypeCtorQuery (deps)
      0b InternChalkImplQuery
      0b InternChalkImplQuery (deps)
      0b InternAssocTyValueQuery
      0b InternAssocTyValueQuery (deps)
leftover: 595mb

Not exactly featherweight, we need to investigate where exactly all those megabytes go.
@matklad matklad added E-hard E-unknown It's unclear if the issue is E-hard or E-easy without digging in fun A technically challenging issue with high impact labels Jan 29, 2020
@matklad matklad mentioned this issue Jan 29, 2020
Merged
@matklad
Copy link
Member Author

matklad commented Jan 29, 2020

Note that TraitSolverQuery (that is, the chalk solver struct itself) went from 0 to 1870mb, which I can't really explain

Ah, this query used untracked read internally, so, just like the intern queries, it wasn't collected.

@bkchr
Copy link
Contributor

bkchr commented Jan 31, 2020

Since #2872 I did not encounter that it eat all my memory, I think that is already a huge step forward :) And makes rust-analyzer usable for me with Substrate :)

@bkchr
Copy link
Contributor

bkchr commented Feb 25, 2020

I'm on 838ad6bcfb2a82c030e18d019b8a06752f0fc828 and something started to eat memory again :( Needed to restart rust-analyzer already twice today.

@lnicola

This comment has been minimized.

Sign in to view
@bkchr
Copy link
Contributor

bkchr commented Feb 25, 2020

The commit I'm on, is before this pr. So, it shouldn't be it.

@bkchr bkchr mentioned this issue Mar 8, 2020
Merged
@bkchr
Copy link
Contributor

bkchr commented Mar 8, 2020

I think it was solved now with: #3502

@matklad matklad closed this as completed Jul 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
E-hard E-unknown It's unclear if the issue is E-hard or E-easy without digging in fun A technically challenging issue with high impact
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lnicola @matklad @bkchr