Builtin semantic variants and language versions #7426
Description
I find that the way Plutus handles the different concepts of version (ledger version, builtin semantic variant, language version) very much over-engineered.
Why not release a new minor/patch language version when new builtins are added to the language or are modified?
I understand that there are some technical subtleties between the language having new constructs (such as case and constr) and adding a builtin function to the language, but I don't think we should make a distinction between them. I like to see the set of builtin functions as the "standard library" of plutus. Every programming language I am familiar with only adds functionality to its standard library through version bumps. One may claim that the language itself does not "change" when a new builtin is added, but from the developer's perspective the interpreter needs to implement that change, which calls for a version bump of some kind.
How we can go forward
I have a plan as to how this could be significantly simplified, while being backwards compatible with the Cardano blockchain.
The first step is to tie all existing builtins to version 1.0.0. So any program that uses plutus 1.0.0 can use all builtins currently available. Also, version 1.0.0 uses the builtin semantic variant 1, where consByteString is infallible. For version 1.1.0, there are no added builtins, but builtin semantic variant 2 is used instead, so consByteString is now fallible. In the future when new builtins are added, the language version is bumped.
Backwards compatibility
So, for language version 1.0.0 we do not change any of the existing behaviour. For language version 1.1.0, we do break programs that were running with --builtin-semantics-variant 1, but this does not affect the blockchain. That is because every program that is written for Ledger versions 1 or 2 (which uses --builtin-semantics-variant 1) needs to use language version 1.0.0. Also, any program that is written for Ledger version 3 (which uses --builtin-semantics-variant 2) needs to use the language version 1.1.0. In both cases, the proposed changes leave the on-chain behaviour intact.
From the plinth documentation, it looks like there are plans to make all builtins available to all ledger versions, and all language versions compatible with all ledger versions. The change I propose above drastically simplifies that process. Once builtins and their semantic variants are tied to a language version, we can simply allow all language versions for any ledger versions.
The abstraction boundary
What I propose draws a clear line between the ledger version and the language version, and removes the need for builtin semantic variants. The ledger version is in charge of the inputs to the program, and the language version is in charge of the behaviour of the program.