Revisit how eager macros are handled

[Veykril]

Right now we do a weird dance:

• We create a macro call for the argument site, immediately expand that into the argument syntax tree
• Then we do what rustc usually does with macro expansion, we walk the tree, expand all macro calls we find and inline those expansion into the tree
• Finally we create a new macro call with a special payload added to it which is this fully expanded tree, which we then intern. This is the actual input to eager macro, its fully expanded input.

For one interning something that changes like this means we'll re-intern on every change, leaking it, but this also feels somewhat weird. In general this is not really how the eager macros in rustc work, they expand their inputs on demand, it is by sheer luck that this tends to be always the full input (except for format_args but there we get around it by using a special custom syntax). It would be nice if we could avoid this weirdness somehow, though the one beneift of this intermediate tracked expansion is that IDE features will simply work for the expanded argument tree. So really the main thing to solve here would be the payload being interned, but salsa doesn't really give us a way to stash this away aside from interning. We could store it within the def-map that computes it, but then we would likely need to eagerly expand the actual invocation as the expand query would then need to access the def map while we are computing the def map forming a cycle

rust-analyzer/crates/hir-expand/src/eager.rs

Lines 44 to 110 in 9fcaab3

	let expand_to = ExpandTo::from_call_site(macro_call);
	// Note:
	// When `lazy_expand` is called, its *parent* file must already exist.
	// Here we store an eager macro id for the argument expanded subtree
	// for that purpose.
	let arg_id = MacroCallLoc {
	def,
	krate,
	kind: MacroCallKind::FnLike { ast_id, expand_to: ExpandTo::Expr, eager: None },
	ctxt: call_site,
	}
	.intern(db);
	#[allow(deprecated)] // builtin eager macros are never derives
	let (_, _, span) = db.macro_arg(arg_id);
	let ExpandResult { value: (arg_exp, arg_exp_map), err: parse_err } =
	db.parse_macro_expansion(arg_id.as_macro_file());
	let mut arg_map = ExpansionSpanMap::empty();
	let ExpandResult { value: expanded_eager_input, err } = {
	eager_macro_recur(
	db,
	&arg_exp_map,
	&mut arg_map,
	TextSize::new(0),
	InFile::new(arg_id.as_file(), arg_exp.syntax_node()),
	krate,
	call_site,
	resolver,
	)
	};
	let err = parse_err.or(err);
	if cfg!(debug_assertions) {
	arg_map.finish();
	}
	let Some((expanded_eager_input, _mapping)) = expanded_eager_input else {
	return ExpandResult { value: None, err };
	};
	let mut subtree = mbe::syntax_node_to_token_tree(
	&expanded_eager_input,
	arg_map,
	span,
	DocCommentDesugarMode::Mbe,
	);
	subtree.delimiter.kind = crate::tt::DelimiterKind::Invisible;
	let loc = MacroCallLoc {
	def,
	krate,
	kind: MacroCallKind::FnLike {
	ast_id,
	expand_to,
	eager: Some(Arc::new(EagerCallInfo {
	arg: Arc::new(subtree),
	arg_id,
	error: err.clone(),
	span,
	})),
	},
	ctxt: call_site,
	};
	ExpandResult { value: Some(loc.intern(db)), err }

[Veykril]

It would be nice if we could change all the eager macros to use some dummy syntax output like we do for format_args, but unfortunately their output is visible to other eager macros.

[davidbarsky]

I wonder if new Salsa's fixed point iteration/speculative execution model can be used to handle this case a bit more nicely.