Add expression parser example demonstrating enums, nested choices, and repetitions

[Copilot]

• Understand the repository structure and existing examples
• Review existing enum and struct derive examples
• Create a new expression example demonstrating:
  • Repetition of anonymous sequences (e.g., term ~ ((mul | div) ~ factor)*)
  • Nested choices (enums for operators like plus|minus)
  • Complex grammar structures commonly found in expression parsers
• Add the example to the derive/examples directory with .pest, .rs files
• Test the new example (11 tests passing)
• Update derive README.md to document the new example patterns
• Run code review
• Run CodeQL security check (no issues found)
• Fix cargo fmt and clippy issues

Summary

This PR adds a comprehensive expression parser example (derive/examples/expression_parser.rs) that addresses the patterns requested in the issue:

Patterns Demonstrated

1. Repetition of anonymous sequences: For term = { factor ~ ((mul | div) ~ factor)* }, shows how to create "tail" structs with manual FromPest implementation to consume (operator, operand) pairs.

2. Nested choices: For operators like (plus | minus) that don't have their own grammar rule, shows how to implement FromPest manually to try each alternative.

3. Parsing into enums: Shows both derived enums (for alternatives with their own rules) and manually implemented enums (for combined operator rules).

Files Added/Modified

• derive/examples/expression_parser.pest - Grammar for arithmetic and comparison expressions
• derive/examples/expression_parser.rs - Complete working example with 11 tests
• derive/README.md - New documentation section explaining the patterns
• examples/csv.rs - Fixed lifetime annotation for clippy

Original prompt

it would be great to have more examples in the documentation. The existing CSV example is helpful, but a limited. For instance, it doesn’t show how to parse into enums or handle more complex grammar structures. Adding a few additional examples would really help newcomers understand how to use pest_ast. Besides that, Now, I have a few cases for which I cannot find a type mapping. i have looked in issues, in the examples of pest-ast and the examples of derive.

A repetition of an anonymous sequence

comparison = { arith_expr ~ (comp_op ~ arith_expr)* }

In #8, I see an example:

struct assigns<'pest>(
  #[pest_ast(outer)] Span<'pest>,
  Vec<struct _1(assign<'pest>)>,
  assign<'pest>,
);

But Vec<struct _1(assign<'pest>)> is not a valid Rust, there are no anonymous structs.

I can try to extract a named struct, but there is no grammar rule to put into pest_ast, and pest_ast is required:

    #[derive(Debug, FromPest)]
    // FIXME: This probably shouldn't work!
    #[pest_ast(rule(Rule::comparison))]
    pub struct ComparisonPart2 {
        pub op: CompOr,
        pub expr: ArithExpr
    }

    #[derive(Debug, FromPest)]
    #[pest_ast(rule(Rule::comparison))]
    // comparison = { arith_expr ~ (comp_op ~ arith_expr)* }
    pub struct Comparison {
        pub expr: ArithExpr,
        pub continuation: Vec<ComparisonPart2>,
        // pub continuation: Vec<(CompOr, ArithExpr)>,
    }

I'm not sure it will work (I haven't finished a mapping for my grammar yet).

A nested choice

arith_expr = { term ~ ((plus|minus) ~ term)* }
How do I map the (plus|minus) part? To an enum like this?

    #[derive(Debug, FromPest)]
    #[pest_ast(rule(Rule::plus))]
    pub struct Plus {
    }

    #[derive(Debug, FromPest)]
    #[pest_ast(rule(Rule::minus))]
    pub struct Minus {
    }

    #[derive(Debug, FromPest)]
    #[pest_ast(rule(Rule::arith_expr))]
    pub enum PlusMinus {
        Plus{ plus: Plus },
        Minus{ minus: Minus },
    }

    #[derive(Debug, FromPest)]
    #[pest_ast(rule(Rule::arith_expr))]
    pub struct ArithExprPart2 {
        pub op: PlusMinus,
        pub term: Term,
    }

    #[derive(Debug, FromPest)]
    #[pest_ast(rule(Rule::arith_expr))]
    // arith_expr = { term ~ ((plus|minus) ~ term)* }
    pub struct ArithExpr {
        pub term: Term,
        pub tail: Vec<ArithExprPart2>
    }

It would be great to have a such example. It would be a bit less great to know it's impossible as of now.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[tomtau]

@copilot cargo clippy and cargo fmt checks fail, can you fix them?

[Copilot]

Pull request overview

This PR adds a comprehensive expression parser example to demonstrate advanced pest_ast patterns that were previously undocumented, specifically addressing user questions about parsing enums, handling nested choices in repetitions, and working with anonymous sequences in grammar rules.

Key Changes:

• New expression parser example showcasing operator precedence, nested choices, and repetitions
• Grammar demonstrating patterns like term ~ ((operator ~ operand)*)
• Updated README with documentation for these advanced patterns

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
derive/examples/expression_parser.rs	Complete expression parser example with extensive documentation, manual FromPest implementations for operators and tail structures, and 11 comprehensive tests covering parsing, operators, chaining, and precedence
derive/examples/expression_parser.pest	Grammar file demonstrating repetitions of anonymous sequences, nested operator choices, and expression precedence patterns
derive/README.md	Added "Advanced Patterns" section documenting how to parse into enums, handle nested choices with manual FromPest, and work with repetitions of anonymous sequences

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The manual FromPest implementation relies on the fact that failed attempts (returning NoMatch) don't advance the iterator. While this is correct according to the FromPest contract, it would be helpful to add a comment explaining this assumption to help future maintainers understand why it's safe to try alternatives sequentially without cloning the iterator first.

[Copilot]

The manual FromPest implementation relies on the fact that failed attempts (returning NoMatch) don't advance the iterator. While this is correct according to the FromPest contract, it would be helpful to add a comment explaining this assumption to help future maintainers understand why it's safe to try alternatives sequentially without cloning the iterator first.

[Copilot]

The FromPest implementation doesn't follow the standard pattern used by the derive macro. While it may work correctly in this specific context due to the grammar structure, it violates the FromPest trait contract which states that NoMatch should leave the iterator unchanged. The implementation calls pest.next() at line 257 before checking if the rule matches, so if it returns NoMatch at line 271, a pair has already been consumed. Consider cloning the iterator first and only updating the original on success, similar to how the derive macro handles this in derive/src/from_pest/mod.rs lines 143-156.

Suggested change

	let pair = pest.next().ok_or(from_pest::ConversionError::NoMatch)?;
	if pair.as_rule() == Rule::comp_op {
	let inner = pair
	.into_inner()
	.next()
	.ok_or(from_pest::ConversionError::NoMatch)?;
	match inner.as_rule() {
	Rule::eq => Ok(CompOp::Eq),
	Rule::neq => Ok(CompOp::Neq),
	Rule::lt => Ok(CompOp::Lt),
	Rule::gt => Ok(CompOp::Gt),
	_ => Err(from_pest::ConversionError::NoMatch),
	}
	} else {
	Err(from_pest::ConversionError::NoMatch)
	}
	// Work on a clone of the iterator and only update the original on success,
	// so that NoMatch leaves the iterator unchanged, as required by FromPest.
	let mut pairs = pest.clone();
	let pair = match pairs.next() {
	Some(pair) => pair,
	None => return Err(from_pest::ConversionError::NoMatch),
	};
	if pair.as_rule() != Rule::comp_op {
	return Err(from_pest::ConversionError::NoMatch);
	}
	let inner = match pair.into_inner().next() {
	Some(inner) => inner,
	None => return Err(from_pest::ConversionError::NoMatch),
	};
	let op = match inner.as_rule() {
	Rule::eq => CompOp::Eq,
	Rule::neq => CompOp::Neq,
	Rule::lt => CompOp::Lt,
	Rule::gt => CompOp::Gt,
	_ => return Err(from_pest::ConversionError::NoMatch),
	};
	// Commit the consumed input only after a successful parse.
	*pest = pairs;
	Ok(op)

[Copilot]

The documentation describes a pattern where you derive FromPest for an enum containing operator variants (lines 20-32), but the actual implementation of AddOp and MulOp uses manual FromPest implementation instead. This inconsistency could confuse readers. Either update the documentation to accurately reflect the manual implementation pattern used, or provide an example that actually uses the derive approach if that's possible.

[Copilot]

Using Result::unwrap here could panic if the input number exceeds i64::MAX. While the grammar ensures only digits are matched, very large numbers will cause parse to fail. For a production parser, consider handling this error gracefully. For an example file demonstrating patterns, you might want to either use a smaller integer type with a clear maximum, add a comment explaining this limitation, or demonstrate proper error handling.

[Copilot]

@copilot cargo clippy and cargo fmt checks fail, can you fix them?

Fixed in commit de036ab. The issues were:

1. Formatting inconsistencies in expression_parser.rs (fixed by cargo fmt)
2. Lifetime annotation in examples/csv.rs changed Span to Span<'_> (pre-existing clippy warning now fixed)