Possibility to set the package version dynamically #6583
Comments
vorner
added
the
C-feature-request
|
I think this could be created as a cargo extension using https://github.com/ordian/toml_edit. You could use https://github.com/killercup/cargo-edit as inspiration. |
|
Actually, I don't want to modify the But yes, you're right, there are certainly less fragile ways to edit |
|
I think there's a ticket for being able to modify options (I guess I'm both Cargo.toml and .cargo/config) via a command, like git config, but I don't know if it extends to temporary overrides. |
|
It's very common use case to specify the version from command line while releasing on a CI server that tracks and auto increments the build number. GNU/make, Maven and Gradle natively support it, manual editing of a build-spec file is not practical and error prone. |
|
|
|
I don't think it is equivalent for my use case. This is for releasing on crates.io and modifying What I want is somewhat completely the opposite. I want to leave |
|
What does
mean? Is it just |
|
I'm not 100% sure there are no other places the version is passed into the compilation, but I guess so. I haven't tried yet (I might), but I guess cargo will set the env var unconditionally and overwrite it even if I pass it from outside. |
|
Maybe we could just change that: only set the environment var if unset. |
|
|
|
Looking at built, it seems to be really nice and useful thing and I'll keep this in mind. But I don't think this will help me here ‒ this doesn't help me push the right version into eg. clap or structopt. |
|
Can you not define the version in clap/structopt in terms of |
|
By default, they take the version from |
|
You're right, sorry. I think the solution is to not override the environment variables, then. |
|
Is there any progress? Also looking for the feature to externally manipulate the package version. (CI is setting the version, Cargo.toml should not be changed). An Env-Variable would be nice: This could also be done for every other part of the Cargo.toml:
The place within the code is here: This should be analogous to https://doc.rust-lang.org/cargo/reference/config.html#environment-variables I could add the implementation if we find a good name for env variables... |
|
Adding another +1 - we're setting the version upon release, and it'd be great to not need to edit Cargo.toml each time |
|
By way of another +1 and showing analogous functionality in other toolchains, the .NET SDK allows for overriding version numbers at the command line with commands like |
|
absolutely same for me. CI and stuff. |
|
This seems to be working on nightly right now: #6699 e.g. |
|
@Humandoodlebug Will not work because --config overrides the cargo configuration, not the build definition in Cargo.toml |
|
Editing the Cargo.toml isn't practical in some scenarios anyway - in Nix we don't really have conventional network access during build phases, and so it's not possible rewrite toml and regenerate lock files to workaround this. I do see that there is a package macro that makes it easy to override the build version, falling back to the Cargo version, but it wouldn't be fun adding this to lots of Rust projects if Cargo could do it: https://docs.rs/git-version/latest/git_version/macro.git_version.html |
|
Very surprising that this is still an open issue in feb 2022. 3 years have passed! probably lost/forgotten in the 1.2k issues. Could we @ some of the active maintainers to get some visibility on this issue? +1 to adding version override in CLI like dotnet. |
|
Is there a way to get |
|
+1 here. I don't understand where the practice of statically specifying a version in a checked in manifest file comes from. I just want to release whatever is already checked in without having to commit again to change any manifests. It's very error prone unless you're using some fancy automated tools. It's so easy to overwrite a version that was already published previously if you release forgetting to change the manifest. Being able to specify the version in the command line and completely omitting the version from the manifest would be my ideal workflow! |
|
I suspect this is a large enough semantic change that this would need a major change proposal or an RFC and need a person to champion the proposal and implementation. It is unlikely for someone else to pick up and do all of that leg work on behalf of someone interested in this work. Things off the top of my head that would need to be figured out
General concerns:
|
|
@epage This is a nice summary to figure out a considerate solution to this problem! Thanks for that! It's a pity that it's unlikely that this effort would come from the core team, but it's understandable. On the other hand, maybe an easier and good enough solution could be adding a flag (or env var) to pass the desired version to cargo so it could ignore the one specified in Cargo.toml? I'm absolutely unaware of cargo's internals, but if there's only one place where Cargo.toml is read to extract the version, then it seems like a pretty easy change to make. In my (somewhat biased) opinion: there should be no static file defining the "current" version of a package at all. This should be determined at release time by looking up the VCS info or elsewhere. |
|
A relevant discussion on Zulip. tl;dr: some build systems need a new version assigned for every build. It would be interesting if they could use build metadata, but there are reasons build metadata doesn't work.
This may be similar to @sunhaitao's use case? |
For cases I just verified,
I think the good balance is: The final package should have static metadata. But they don't all have to be hard-coded in the source. Some of them should be able to be decided on publish. Allowing all of them being so dynamic seems unnecessary. |
Actually, we align the {MINOR} part to "How many days have passed since we started working on this {MAJOR} version". If we pause works on a package for a while, the {MINOR} part will jump several numbers on resuming. And we don't publish all those versions to end users. Many of them only exist on developer machines. Some on build machines. Only few of them are released out. |
|
It is similar on frequently assigning a version. But on build machines, we only dare to touch the {version core} part on each commit. Anyway, changing the version a little on each build is a important use case for this feature. |
|
I'm just joining this discussion now. I thought better than responding to individual comments which are sometimes quite old, the best thing I could do was state my case mostly from scratch. Sorry if anyone feels like rehashes old ground. My use-caseIn the case of Rust, I'm an application author. I'm not publishing libraries. I want to be able to specify the version from some external source, and I don't want to edit any files. Certainly not manually, but ideally not even programmatically. My usual workflow - one I've successfully used in other ecosystems - is that application versions are completely managed by git tags. In the source code and config files, you will never find a version (except possibly in a readme). When I create a release tag (i.e. I don't want to have to manually edit a TOML file, commit it, and then tag that commit. In the past, I've worked for companies where QA liked to sign-off on a precise commit hash. If QA test a version and then the team later decides "let's release that as 1.0.0", then with Cargo I have to add more commits to specify the version, and the hash they signed-off on doesn't match any more. Another problem is that if a version is specified by both a tag and a TOML file, then it's possible for them to be out of sync. What if my tag is There's at least two workarounds I can think of: I would like some equivalent command to What Java doesI come from a Java background and Maven supports this, with what they call "CI friendly versions". It's explained here https://maven.apache.org/guides/mini/guide-maven-ci-friendly.html You can see a personal example where I use this pattern here: POM, somewhat equivalent to Cargo.toml: https://github.com/michaelboyles/s3-maven-extension/blob/develop/pom.xml#L9C5-L9C35 GitHub build: https://github.com/michaelboyles/s3-maven-extension/blob/develop/.github/workflows/build.yml#L38 Maven has multi-module support, where each module may theoretically have a unique version, but they don't provide any way to specify those versions independently via CLI: you only get one CLI-settable "revision". I've never seen anyone complain that that wasn't sufficient. So maybe the answer to "How do we pass in versions, especially when running against multiple packages?" is "you don't, and that's fine". Since Maven's POM is itself an uploadable artifact, there's an additional step required to erase the placeholders to produce a final artifact POM that's suitable for consumers (e.g. replace |
|
Why do you need cargo to know about your version at all? You want to manage the version outside cargo ( I posted a solution further up that does what you want and works well for applications. It's even cleaner now than when I posted it, since now the entire |
|
@faern assuming that was a reply to me I didn't say I need it. I ended up here after fiddling unsuccessfully with I don't currently do this with Rust, but with Maven I publish both libraries and apps into Sonatype Nexus. There's no real reason the apps have to be stored there, because Maven is never going to download them, but it's a convenient structured storage that prevents overwriting a release version, and we already need it for libraries anyway. I could foresee wanting to This would give application authors a convenient, consistent and idiomatic way to specify versions, rather than having to invent something for themselves. That seems desirable whether or not it's a necessity |
for instance because with maturin your wheel will end up with the version number that cargo gives it ? Specific example I know, but I landed in this thread with the exact same need as many others it seems. Editing the cargo.toml (even programmatically) makes little practical sense, while versioning artifacts is recognized good practice ?
Thanks for posting it, it's clever, but still feels like fighting against the tooling here, not exactly straightforward ? |
|
There's a bit of buck-passing going on here. The maturin project says they'll only use the cargo version, and won't provide an override, so we must use the cargo version to set the package version, yet the cargo project says that you should use a separate version number, not the cargo version number. The two approaches aren't compatible. |
In my case it works fine eventually, I was mentioning this as an example really. There are a bunch of examples in this thread showing that while the file based approach makes sense, modern CD is easier to set up when version can be flag based. That's pretty much it, CD pipelines are not file based with respect to version, typically. I think the collision is here more than with Maturin (which just delegates this upstream). |
|
@epage You proposed plugin API before. It would definitely help to ease the situation, introducing algorithms which could be tested faster and easier outside core code. Currently bumping version and release a new one can't be done in a single workflow with no human interaction. Current workflow is following:
At the last step we can't be sure that between first and last steps there were no commits/merges This workflow
|
|
Let's convert my previous comment to company costs. As shown below they at least 2-5 times higher per each release, thus cheaper alternatives like "write on another language" or "fork and maintain fork of cargo" are way cheaper than current situation. In a medium plus size company normal releases of apps happen usually once in 2-4 weeks per app/library. This cost has also highly negative impact on critical issues on production, when developer may need to fix an issue at at night. Times on localhost for comparison for a small library I've wrote recently. Everyone can scale them to CI and their size of app/test coverage
when build system accepts CI version:
when build system doesn't accepts CI version:
Additional run of tests including time of tester team is costly for a company and it's easier to reject Rust as a language of choice or override the issue by writing and maintaining their release tool instead of using cargo itself. |
|
I'm pretty sure there should be no more than one "source of knowledge" for the crate, such as version. Imho, this is a wrong direction and will entail serious problems for cargo and the entire ecosystem. Next step is |
if cargo would accept such parameter, it would be enough to fix the issue. As shown above |
Are you using the version for lookups in an internal or public registry or for the marketing version? If for the marketing version, could you explain why you want to use |
|
Private registry is considered as primary target and public registry for open source parts. My company rejected proposition to rewrite part of apps and libs to rust due to fact, that cargo can't be aligned to the process. Forking cargo for packaging was considered, but rejected as company wasn't sure if they want to invest in Rust. Some of my personal projects aren't released to the public registry because of that exact reason as well. Searching for a acceptable solution with automation took me too long. Probably, I'll release them without automation, even this not how I prefer. |
|
Shout out to @psFried 's I am building CLIs in AzureDevops and we use Date based versions, and this is working to at least get the version reported by the CLI itself to what we want. Update: PSA from @eirnym that this doesn't influence // build.rs
fn main() {
// Set the version to the build ID if it exists
if let Ok(version) = std::env::var("BUILD_BUILDID") {
println!("cargo:rustc-env=CARGO_PKG_VERSION={}", version);
}
println!("cargo:rerun-if-env-changed=BUILD_BUILDID");
} |
|
@kyle-rader-msft works well for marketing version, but it won't fool This article describes how to extend |
|
I've gave a thought on what possible options do we have and wrote following proposition. Please, comment and propose your solutions Possible solutionsHere I describe various possible solutions, which maybe combined together. All solutions must utilize semver to verify input and reject the rest. In following document I only respect Git SCM, as I don't take solution "which works everywhere with everything from day zero" as a good practice. Environment variableEnvironment variable to set the version. Advantages of this solution:
Disadvantages of this solution:
Command line parameterSimilar advantages and advantages to an environment variable solution, but it isn't scalable. build.rsUsing existing e.g. Cargo reads Cargo.toml, sets up variables, run Advantages of this solution:
NOTE: git tags may be fetched without fetching whole repository as shown in https://github.com/actions/checkout. Disadvantages of this solution:
Additional cargo.rsVery similar to Disadvantages of this solution:
Scm version in Cargo.toml, e.g.
|
|
My personal opinion on what I've wrote above (cut out this part, as it is not main part of implementation proposition)
|
Hmm... Trying to come up with an interims solution myself that is satisfying to me makes my brain hurt... Mainly because I'm so used to the conventions of build tools of other languages and am trying to apply the same way of thinking to Maybe it would be an option to just define a very simple interface for an external mechanism and just require that mechanism to provide any valid semver string? The external mechanism could be identified by a simple pattern that does not collide with the semver pattern, like prefixing with Assuming a binary, or a file with a shebang (therefore executing through the shell environment), something like: Assuming a command, something like: In the end, all I want is for my Git working tree to stay clean, after I've applied a tag. If versioning was handled completely outside of |
|
@eirnym: You mention an environment variable. Read by which process? Set by which one? Do you envision one that can be set in |
|
Currently only marketing version can be set by Environment will remove requirement to always have Why library first? Because if we'd have a library for Due to |
@eirnym, I see what you mean. I think I conflated two separate issues, pardon... I was describing a separate use-case: to set the version dynamically and forward it to mechanisms like |
|
I see misunderstanding as well. "Markeing version", as has been used by @epage is set the version for a binary which is never published. This version can be set at this moment by passing variable to rustc via I'm concentrating and commenting exclusively on publishing version, which is propagated in resulting |
|
@eirnym thank you for taking the time to enumerate the different options! Could you update your solution alternatives to address the considerations from
As for |
|
Thank you @epage, I'll take a look and answer all questions as soon as possible. |
and others
Maintainer notes
package.version, see https://github.com/killercup/cargo-editHello
I'm building a rust application using internal teamcity CI job. I want the build number to be part of the version of the binary (so eg.
./app --versionknows from which build it came).The only way I found so far is to let the build first edit the
Cargo.toml(sed -i -e 's/^version = .*/version = "%build.version%"/' Cargo.toml), which seems ugly and fragile.It would be great if I could somehow override the version from
Cargo.tomlthrough the command line ‒ either as a parameter or an environment variable.Would something like that make sense? Are there plans to support it?
The text was updated successfully, but these errors were encountered: