Structure rust-guidelines to make code easily accessible

[olsonjeffery]

This probably dovetails with #17; I'm specifically interested in having all of the code in rust-guidelines be easy reachable/buildable/etc, in order easily integrate with downstream analysis tools (specifically rustfmt).

In my case, I've been working on pcwalton/rustfmt a bit lately and I feel like it's finally nearing a place where it can focus on chasing down comforming to rust-guidelines (at least the style portion).

I'd like to be able to take rust-guidelines as a dependency from rustfmt, build rustfmt, and then parse every code snippet in the style section and fail if any of rustfmt's output differs from the input as it exists in the style snippets. Right now this would involve a manual process of running through every .md file in the style directory to pull out the rust snippets, which I'd rather avoid if at all possible. Does why this is not really desirable make sense and give an idea of how we could arrange the code/docs in rust-guidelines to make it more amenable to integrating with other tools/processes?

Anyways, I see #17 is already there, but there's not much in the way of commentary around that. So I'd like to start that conversation about how we can approach this. In any case, having every code snippet pass muster with rustc is a desirable goal of rust-guidelines, and pursuing this goal would also serve the purposes I have w/ rustfmt.

Please do let me know if there's anything I can contribute towards this goal. I could also open a topic on the discussions board/listserv.

[nathantypanski]

You'd need some kind of obvious way to pick out the code snippets. Preferably besides just markdown delimiters, like you said.

I brought up the idea of literate rust code on the mailing list around a month ago. I started working on a proof-of-concept, but never really built up any momentum from me. Keep in mind this code is very very simple and has not really seen much love from me; I've been planning on expanding it when I find some more time. The idea was eventually to start using rustdoc and rustc as libraries to integrate it nicely with the toolchain.

But I think the whole literate programming format is completely obvious for this kind of application. You could run the literate files like they're regular code, but write markdown around them. It's the best of both worlds when you have code that serves as supplementary information to the text just as much as the text supplements the code.

Of course, there might be other ideas on how to achieve this.

[olsonjeffery]

@nathantypanski Thanks for the feedback. I understand the benefit of literate programming, but don't think it's appropriate to the particular use case I was describing.

It's valuable to be able to break snippets of style guidance down to their individual pieces and take them, one-by-one. A single, literate-programming style document works against that goal. Insofar as I want to lower the barrier to analyzing the snippets of rust-guidance code in downstream tools, I personally am only interested in approaches that keep the snippets easily separable.

[nathantypanski]

Right, that's the whole point of literate programming. When it comes to "keeping snippets separable" - that bit's easy. As in, here's an implementation in sed. If you mean "can I separate individual segments of code from other ones on the same page" - I'd say that's still facilitated by some of the necessary functions and document structure of any LP text. There's some surrounding infrastructure required, yeah. Because people will only bother to produce this easily-separable code if they're actually working within a LP environment to begin with. Which, as of now, nobody writing Rust code is.

But, I mean it would solve

I'm specifically interested in having all of the code in rust-guidelines be easy reachable/buildable/etc,

OTOH, in spite of advancing some "political" ideals here (I think literate programming deserves some attention, since it's useful when writing instructional manuals and documentation), it probably isn't very hard to parse the markdown snippets in this code.

I mean, can't you build a finite state machine that will:

1. Upon encountering a block of \n[ \t]+\n```.*\n, start reading the following strings as Rust code
2. Upon reaching a subsequent \n```\n[ \t]+\n block, stop reading as Rust code.
3. Interpret all sequences of the form \n[ \t]+\n followed by four spaces, then another \n[ \t]+\n block as another form of code block.

In other words, isn't Markdown syntax (in particular, this lone piece of markdown syntax) easy enough to parse that this isn't especially an issue?

[aturon]

@olsonjeffery Thanks for asking about this -- I would love for rustfmt to come into alignment with the guidelines.

I just moved rust-guidelines to a new homegrown tool, rustbook which is just a thin layer on top of rustdoc. This should make it easier to start testing the snippets, and we can probably also harness rustdoc to pull out the code snippets as well (since it has to do that for tests).

For now, rustbook can only render the book -- it can't test it. But I will add testing very soon, and then work to get the code snippets in the guidelines to actually pass rustc.