Skip to content

issue-130 copy contents related x.py from rust-forge to rustc-guide #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 29, 2018
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
- [About this guide](./about-this-guide.md)
- [About the compiler team](./compiler-team.md)
- [How to build the compiler and run what you built](./how-to-build-and-run.md)
- [Build and Install distribution artifacts](./build-install-distribution-artifacts.md)
- [Documenting Compiler](./compiler-documenting.md)
- [Coding conventions](./conventions.md)
- [Walkthrough: a typical contribution](./walkthrough.md)
- [The compiler testing framework](./tests/intro.md)
Expand Down
28 changes: 28 additions & 0 deletions src/build-install-distribution-artifacts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Build distribution artifacts

You might want to build and package up the compiler for distribution.
You’ll want to run this command to do it:

```bash
./x.py dist
```

## Other Flags

The same flags from build are available here.
You might want to consider adding on the -j flag for faster builds
when building a distribution artifact.

```bash
-j, --jobs JOBS number of jobs to run in parallel
```


# Install distribution artifacts

If you’ve built a distribution artifact you might want to install it and
test that it works on your target system. You’ll want to run this command:

```bash
./x.py install
```
11 changes: 11 additions & 0 deletions src/compiler-benchmarking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Benchmarking rustc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... @nnethercote are these benchmarks maintained and still useful?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't heard of these. I just tried ./x.py bench, it looks like it's basically the same as cargo bench, i.e. it runs #[bench] unit benchmarks. That might be useful for benchmarking certain components within the compiler (mostly libraries in std) but it doesn't give any info on the overall speed of the compiler. Also, I got compile errors for a bunch of these, so it looks like they aren't being run regularly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajcspsg Perhaps we should remove this section/file altogether then? It seems to be outdated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is removed.


This one is a easier compared to the others.
All you’re doing is running benchmarks of the compiler itself
so it’ll build it and run the one set of benchmarks available to it.
The command is:

`./x.py bench`

Benchmarking lacks `--no-fail-fast` flag that `test` command has.

60 changes: 60 additions & 0 deletions src/compiler-documenting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Documenting rustc

You might want to build documentation of the various components
available like the standard library. There’s two ways to go about this.
You can run rustdoc directly on the file to make sure the HTML is
correct, which is fast. Alternatively, you can build the documentation
as part of the build process through x.py. Both are viable methods
since documentation is more about the content.

## Document everything

```bash
./x.py doc
```

## If you want to avoid the whole Stage 2 build

```bash
./x.py doc --stage 1
```

First the compiler and rustdoc get built to make sure everything is okay
and then it documents the files.

## Document specific components

```bash
./x.py doc src/doc/book
./x.py doc src/doc/nomicon
./x.py doc src/doc/book src/libstd
```

Much like individual tests or building certain components you can build only
the documentation you want.

## Document internal rustc items
By default rustc does not build the compiler docs for its internal items.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidtwco Is this still true? How is the rustc rustdoc built?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compiler documentation is not built by default - there's a config.toml flag for that. But, when enabled, compiler documentation does include internal items.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajcspsg Perhaps we can replace the first paragraph in this section (and the cp config.toml.example ... snippet) with what davidtwco wrote in the comment directly above?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is done

Mostly because this is useless for the average user. However, you might need
to have it available so you can understand the types. Here’s how you can
compile it yourself. From the top level directory where x.py is located run:

```bash
cp config.toml.example config.toml
```

Next open up config.toml and make sure these two lines are set to true:

```bash
docs = true
compiler-docs = true
```

When you want to build the compiler docs as well run this command:

```bash
./x.py doc
```

This will see that the docs and compiler-docs options are set to true
and build the normally hidden compiler docs!
91 changes: 91 additions & 0 deletions src/how-to-build-and-run.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ debuginfo-lines = true
use-jemalloc = false
```

### what is x.py?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "what" -> "What"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done.


x.py is the script used to orchestrate the tooling in the rustc repository.
It is the script that can build docs, run tests, and compile rustc.
It is the now preferred way to build rustc and it replaces the old makefiles
from before. Below are the different ways to utilize x.py in order to
effectively deal with the repo for various common tasks.

### Running x.py and building a stage1 compiler

One thing to keep in mind is that `rustc` is a _bootstrapping_
Expand Down Expand Up @@ -80,6 +88,52 @@ compiling `rustc` is done in stages:
can build the libraries with the stage2 compiler. The result ought
to be identical to before, unless something has broken.


#### Build Flags

There are other flags you can pass to the build portion of x.py that can be
beneficial to cutting down compile times or fitting other things you might
need to change. They are:

```bash
Options:
-v, --verbose use verbose output (-vv for very verbose)
-i, --incremental use incremental compilation
--config FILE TOML configuration file for build
--build BUILD build target of the stage0 compiler
--host HOST host targets to build
--target TARGET target targets to build
--on-fail CMD command to run on failure
--stage N stage to build
--keep-stage N stage to keep without recompiling
--src DIR path to the root of the rust checkout
-j, --jobs JOBS number of jobs to run in parallel
-h, --help print this help message
```

One thing to keep in mind is that `rustc` is a _bootstrapping_ compiler. That
Copy link
Member

@mark-i-m mark-i-m Sep 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already discussed in the previous section. I think we don't need lines added 114-136.

EDIT: i.e. from "One thing to keep in mind" to "unless something has broken".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done.

is, since `rustc` is written in Rust, we need to use an older version of the
compiler to compile the newer version. In particular, the newer version of the
compiler, `libstd`, and other tooling may use some unstable features
internally. The result is the compiling `rustc` is done in stages.

- **Stage 0:** the stage0 compiler can be your existing
(perhaps older version of)
Rust compiler, the current _beta_ compiler or you may download the binary
from the internet.
- **Stage 1:** the code in your clone (for new version)
is then compiled with the stage0
compiler to produce the stage1 compiler.
However, it was built with an older compiler (stage0),
so to optimize the stage1 compiler we go to next stage.
- **Stage 2:** we rebuild our stage1 compiler with itself
to produce the stage2 compiler (i.e. it builds
itself) to have all the _latest optimizations_.
- _(Optional)_ **Stage 3**: to sanity check of our new compiler,
we can build it again
with stage2 compiler which must be identical to itself,
unless something has broken.

For hacking, often building the stage 1 compiler is enough, but for
final testing and release, the stage 2 compiler is used.

Expand Down Expand Up @@ -134,6 +188,32 @@ build`) has quite a few more steps:

<a name=toolchain></a>

### Build specific components

Build only the libcore library

```bash
> ./x.py build src/libcore
```

Build the libcore and libproc_macro library only

```bash
> ./x.py build src/libcore src/libproc_macro
```

Build only libcore up to Stage 1

```bash
> ./x.py build src/libcore --stage 1
```

Sometimes you might just want to test if the part you’re working on can
compile. Using these commands you can test that it compiles before doing
a bigger build to make sure it works with the compiler. As shown before
you can also pass flags at the end such as --stage.


### Creating a rustup toolchain

Once you have successfully built rustc, you will have created a bunch
Expand Down Expand Up @@ -263,3 +343,14 @@ This allows you to do "jump-to-def" with whatever functions were around when
you last built, which is ridiculously useful.

[etags]: https://github.com/nikomatsakis/rust-etags

### Cleaning out build directories

Sometimes you need to start fresh, but this is normally not the case.
If you need to run this then rustbuild is most likely not acting right and
you should file a bug as to what is going wrong. If you do need to clean
everything up then you only need to run one command!

```bash
> ./x.py clean
```
21 changes: 21 additions & 0 deletions src/tests/running.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,24 @@ This is much faster, but doesn't always work. For example, some tests
include directives that specify specific compiler flags, or which rely
on other crates, and they may not run the same without those options.

### Run only the tidy script
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps move these to the end of the "Running a subset of the test suites" section above (in the same file).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done.

```bash
> ./x.py test src/tools/tidy
```
### Run tests on the standard library
```bash
> ./x.py test src/libstd
```

### Run tests on the standard library and run the tidy script
```bash
> ./x.py test src/libstd src/tools/tidy
```

### Run tests on the standard library using a stage 1 compiler
```bash
> ./x.py test src/libstd --stage 1
```

By listing which test suites you want to run you avoid having to run tests for
components you did not change at all.