The server seems to fail to generate the documentation page BigUInt.

[piaoyh]

Crate name

cryptocol

Build failure link

https://docs.rs/cryptocol/latest/cryptocol/number/big_uint/struct.BigUInt.html#struct.BigUInt

Requested RAM limit

No response

Requested timeout

No response

Requested number of targets

No response

Why your crate needs the resource increases

Up to the version 0.8.4 or earlier, if you click the tap 18 versions on the website (https://crates.io/crates/cryptocol) and then you click the item 0.8.4 as shown bellow:

then the browser will navigate to the page (https://crates.io/crates/cryptocol/0.8.4). Then if you click the link of docs.rs/cryptocol/0.8.4 on the right side of the cryptocol introduction page of crate.io (https://crates.io/crates/cryptocol) as shown below:

then the browser navigates to the main page of the crate cryptocol through the link (https://docs.rs/cryptocol/0.8.4/cryptocol/). There, if you click the link BigUInt (https://docs.rs/cryptocol/0.8.4/cryptocol/number/big_uint/struct.BigUInt.html#struct.BigUInt) to the documentation page of BigUInt as shown below:

then the server generates the documentation page of BigUInt which the link (https://docs.rs/cryptocol/0.8.4/cryptocol/number/big_uint/struct.BigUInt.html#struct.BigUInt) is pointing to, and the browser shows the documentation page of BigUInt (https://docs.rs/cryptocol/0.8.4/cryptocol/number/big_uint/struct.BigUInt.html#struct.BigUInt) correctly and successfully without any problem as shown below:

That is fine!!!

However, from the version 0.8.5 on, when you click the link of BigUInt of the cryptocol introduction page of crate.io (https://crates.io/crates/cryptocol) as shown below:

then the browser navigates to the documentation page of BigUInt through the link (https://docs.rs/cryptocol/latest/cryptocol/number/big_uint/struct.BigUInt.html#struct.BigUInt), but the server does not generate the documentation page of BigUInt which the link (https://docs.rs/cryptocol/latest/cryptocol/number/big_uint/struct.BigUInt.html#struct.BigUInt) is pointing to, and the browser shows the page as shown below:

in which it is written as follows:
The requested resource does not exist
no such resource.

Also if you click the link of docs.rs/cryptocol/0.8.5 on the right side of the cryptocol introduction page of crate.io (https://crates.io/crates/cryptocol) as shown below:

then the browser navigates to the main page of the crate cryptocol through the link (https://docs.rs/cryptocol/0.8.5/cryptocol/). There, if you click the link BigUInt (https://docs.rs/cryptocol/0.8.5/cryptocol/number/big_uint/struct.BigUInt.html#struct.BigUInt) to the documentation page of BigUInt as shown below:

then the server generates the documentation page of BigUInt which the link (https://docs.rs/cryptocol/0.8.5/cryptocol/number/big_uint/struct.BigUInt.html#struct.BigUInt) is pointing to, but the server does not generate the documentation page of BigUInt which the link is pointing to, but the browser shows the page as shown below:

in which it is written as follows:
The requested resource does not exist
no such resource.

Actually, the source file '/src/number/big_uint.rs' has more than 40,000 lines as shown below and it will get more as I develop more. I don't know whether or not it reaches the limitation.

Please dig this problem and let me know the reason if the reason is in my work. And if the reason of the problem is in the server, please fix it.

If you need more information about this problem, please let me know.

Many blessings

[syphar]

Hi, sorry for the delay here, was swamped with daywork, and the other docs.rs production issues :)

I'm not sure what the problem is, but will investigate.

Btw, a good starting point is sometimes the archive we are generating in the build:
https://docs.rs/about/download

Which is where the files are pulled out by the webserver.

[syphar]

so, checking the archive I can see the issue, and we could definitely improve the error message here.

Our webserver has a configured max file size of 50 MiB per HTML-file, which is already huge for every browser to handle.

And struct.BigUInt.html in this case has nearly 60 MiB, which (on my machine) crashes chrome, and takes annoyingly long to open in Firefox.

Since we don't stream the files yet internally in the server, we won't increase this limit for now. Streaming the content is on the TODO list, but other topics are before that.

I assume when you build the docs locally you will have similar filesizes, so the solution is that you reduce the doc file size for this file either by reducing docs themselves, or splitting the file into several files.

cc @GuillaumeGomez or is there a nicer way?

[syphar]

So to wrap it up: we are generating the documentation page, but it's too big to serve

[GuillaumeGomez]

cc @GuillaumeGomez or is there a nicer way?

I'm surprised that doc pages can be so big. O.o

And absolutely no clue. Apart from removing docs/items I don't really see how to reduce the page size.

[piaoyh]

I wrote email to Denis about this matter but I haven't gotten any reply from him. So, I write it again here.

After I generated the documentation by "cargo doc" on my local machine, I found that the struct.BigUInt.html is only 4.6 MB on my machine.
Please see the following.

Are we talking about the same file?
struct.BigUInt.html is located in '.../doc/cryptocol/number/big_uint'. Did you mean the same file '.../doc/cryptocol/number/big_uint/struct.BigUInt.html'?
Whole documentation files are all 39.2 MB only. Please see the following.

Strange! Why is '.../doc/cryptocol/number/big_uint/struct.BigUInt.html' so big in the server while it is only 4.6 MB on my machine?

I think that the server does not generate the webpage as cargo does.

Please report this problem to the team of crates.io webserver developers.
Merry Christmas.

[syphar]

I'll answer out of order.

Please report this problem to the team of crates.io webserver developers.

They don't have anything to do with this, it's either this (docs.rs) or the builder (rustdoc).

I wrote email to Denis about this matter but I haven't gotten any reply from him. So, I write it again here.

Sorry, we don't provide private support for anyone, the channels are either github issues, or zulip threads.

After I generated the documentation by "cargo doc" on my local machine, I found that the struct.BigUInt.html is only 4.6 MB on my machine.

back to the issue, super interesting :)

To build docs the same we we do, you have to use a nightly toolchain, and set some build parameters.

So first you should try to build with nightly (cargo +nightly docs) and check the output. These docs you then can also open and see what contributes to the size.

On top of that we set some arguments we need for hosting, you can come quite near to these by using cargo docs-rs and then call

$ cargo +nightly docs-rs 
...

to check the output.

[piaoyh]

Strange!
When I execute cargo +nightly docs, it keeps running but never ends. It looks that the cargo falls into infinite loop. Frankly speaking, I don't know what is its reason or what causes cargo to fall into infinite loop. I have used cargo +nightly docs for the first time.

Though it keeps running, when I check the size of struct.BigUInt.html, it is found 4.6 MB. Its size does not grow. And, the whole size of the folder doc is 39.4 MB as follows.

Even though I refreshed several times with some time interval, the size didn't grow. It means cargo is doing something else rather than generating struct.BigUInt.html or any other html files.

Do you think that my source code including documentation is guilty or that cargo does something wrong when it is executed with the arguments +nightly and docs especially against big source codes? I agree that my documentation especially for BigUInt is big compared to other source codes of other developers, but the size of its source code BigUInt.rs that includes documentation is only 2.1 MB as follows.

Would you please report this issue to the team of cargo developers? I dare to suspect that cargo does not work well (or properly) with the arguments +nightly and docs especially against big source codes. It works fine with the arguments doc even against big source codes such as 2.1 MB-sized file.

Anyway, merry Christmas to Denis Cornehl and others!

[syphar]

When I execute cargo +nightly docs, it keeps running but never ends. It looks that the cargo falls into infinite loop. Frankly speaking, I don't know what is its reason or what causes cargo to fall into infinite loop. I have used cargo +nightly docs for the first time.

how long did you try running it?

Would you please report this issue to the team of cargo developers? I dare to suspect that cargo does not work well (or properly) with the arguments +nightly and docs especially against big source codes. It works fine with the arguments doc even against big source codes such as 2.1 MB-sized file.

This is not a cargo issue.

My current guess is:

• there are some new features in rustdoc in nightly that are not on stable
• with a source file that big building the docs takes more time (docs.rs has it), and creates a huge doc file.
• my guess it that if you would let the local build run for an hour at least, you will see the same result

Generally source files that big are difficult to handle for many systems, so making it smaller would be a good optimization in any case.

@GuillaumeGomez might this be a size regression? Was there much added compared to stable?

[GuillaumeGomez]

The only big change in rustdoc I'm aware of is the add of the merged doctests, so it shouldn't impact things here. However there were a lot of small changes, so maybe one (or more) of them is messing things up.

We need to have something to debug to figure out what's wrong. ^^'

[piaoyh]

How long did you try running it?
I ran it for about twenty minutes or a little bit less.
I didn't let it run for an hour or that long.

Still, I feel it strange about why it takes so irrationally long and generates irrationally big size of file. BigUInt.rs of version 0.8.4 is 2.9 MB in size while BigUInt.rs of version 0.8.5 is 4.0 MB. The change (1.1 MB) in size makes huge difference. There must be a size threshold that cargo cannot deal with files.

[GuillaumeGomez]

I think it's an infinite loop, the interesting thing would be to find a small case triggering this infinite loop so the rustdoc team could fix it and have a regression test.

[piaoyh]

Then, would you please report this issue to the team of cargo developers or whatever proper team?

I think, the fact that cargo doc works for the same source codes but cargo +nightly docs does not work for the same source code indicates something wrong in cargo. If my codes with documentation are guilty, both cargo doc and cargo +nightly docs should not work.

Thanks to Guillaume Gomez for your reporting this issue to the proper team of cargo development in advance.

Merry Christmas to everybody