Add source package for file-based programs support #51373

RikkiGibson · 2025-10-20T23:10:51Z

This package will be used to support dotnet/roslyn#80575.

Some things I haven't figured out yet:

~~GracefulException probably won't work in the source package.~~
- Adding a compilation symbol which simply declares this type when we are not in context of dotnet cli
JSON generator error in RunApiCommand.cs.
~~Failing to include MSBuildUtilities.cs in the source package. See Microsoft.DotNet.FileBasedPrograms.projitems.~~ It seems like a .cs file which is "out of tree" doesn't make it into the package. I'm unsure if that is expected. @tmat may have some suggestion for how to address this part.
- Working around this by simply making another copy.

@jjonescz would be interested to know any thoughts you have on the above bullets or the TODOs in source.

RikkiGibson · 2025-10-21T03:05:17Z

src/Cli/dotnet/Commands/Run/Api/RunApiCommand.cs

 }

+// TODO: getting the following error, which is not suppressible with pragma.
+// error SYSLIB1225: The type 'Encoding' includes the ref like property, field or constructor parameter 'Preamble'. No source code will be generated for the property, field or constructor. (https://learn.microsoft.com/dotnet/fundamentals/syslib-diagnostics/syslib1225)


It looks like it is complaining about System.Text.Encoding.Preamble. But AFAIK, I only changed where some of the types involved are declared, not what their members are.

I managed to "debug" this by commenting out some of the attributes and then members on various types.

It looks like this is ultimately coming through via:

[JsonSerializable(typeof(RunApiOutput))] to

class Project : RunApiOutput to

ImmutableArray<SimpleDiagnostic> Diagnostics to

Position SimpleDiagnostic.Location to

SourceFile SimpleDiagnostic.SourceFile to

SourceText SourceFile.Text to

Encoding SourceText.Encoding

... and, it looks like the release/10.0.1xx branch version of SimpleDiagnostic does not have SourceFile on it. So, I think moving to that "version" of the declaration would probably make the generator error go away.

It looks like I copied the implementation from the wrong branch here. @jjonescz is release/10.0.1xx the correct branch to be using as a starting point for changes here? I assume we can't get this source package work into there as it is taking servicing fixes only. Is there some step we neglected to take in order to make sure the changes going into the release branch make their way into main?

release/10.0.2xx should be open to normal changes and up to date. I'm using that one for run-file development now. main haven't received updates from release/10.0.1xx since September 20.

This is the correct mindset for navigating the SDK repo 👍

Thanks! I will plan on updating this based on release/10.0.2xx branch, and changing target to the same. Before I open for “full” review, I will try to set up the history so that the first commit is a pure move.

jjonescz · 2025-10-22T13:35:36Z

src/Cli/Microsoft.DotNet.FileBasedPrograms/AppDirectiveHelpers.cs

+internal static class AppDirectiveHelpers
+{
+
+#pragma warning disable RSEXPERIMENTAL003 // 'SyntaxTokenParser' is experimental


This suppression shouldn't be needed. I'm removing it in #51410

jjonescz · 2025-10-22T13:36:38Z

src/Cli/Microsoft.DotNet.FileBasedPrograms/AppDirectiveHelpers.cs

+
+namespace Microsoft.DotNet.FileBasedPrograms;
+
+internal static class AppDirectiveHelpers


I think the official term might be "file-level directives", not "app directives" as the latter seems too generic, although in the context of namespace "FileBasedPrograms" it's fine

Yes I think changing term to match the spec will be best here. I can make the same change on the editor features side. https://github.com/dotnet/sdk/blob/main/documentation/general/dotnet-run-file.md#directives-for-project-metadata

jjonescz · 2025-10-22T13:38:06Z

src/Cli/Microsoft.DotNet.FileBasedPrograms/ExternalHelpers.cs

+namespace Microsoft.DotNet.FileBasedPrograms;
+
+/// <summary>
+/// When targeting netstandard2.0, the user of the source package must "implement" certain methods by declaring members in this type.


Missing </summary> closing tag

jjonescz · 2025-10-22T13:43:29Z

src/Cli/Microsoft.DotNet.FileBasedPrograms/MSBuildUtilities.cs

I'm not sure why copying this file is needed. Does adding <Compile> to the package project not work? I see you are saying "It seems like a .cs file which is "out of tree" doesn't make it into the package." but I've recently created a source package where all the files come from a different directory and it works fine:

https://github.com/dotnet/roslyn/blob/main/src/NuGet/Microsoft.CodeAnalysis.BuildClient.Package/Microsoft.CodeAnalysis.BuildClient.Package.csproj

Although I wasn't using shproj and projitems, perhaps that's the problem?

In my case I tried adding a Compile item to the .projitems. I'll take a look at the build for the source project you linked and try to work out what the difference is.

jjonescz · 2025-10-22T13:44:23Z

src/Cli/Microsoft.DotNet.FileBasedPrograms/ExternalHelpers.cs

+    {
+    }
+
+    public GracefulException(string? message, string? todo2) : base(message)


The todo2 parameter is unused. Can we just remove this overload?

This overload is being called in the implementation. Perhaps when compiling against the proper version, the GracefulException(string, params string[]) overload is used. But I think it would be necessary to do more work to get rid of the usages, in order to delete this overload.

I think the work to drop usage of this type in the source package might be worthwhile, I am just trying to push it out until after I can get the end-to-end working. I would also be interested in your thoughts on that.

I guess we can keep the overload and just make sure the second parameter isn't dropped; e.g., using string.Format in the implementation?

tmat · 2025-10-22T16:51:51Z

src/Cli/Microsoft.DotNet.FileBasedPrograms/ExternalHelpers.cs

+}
+
+#if FILE_BASED_PROGRAMS_SOURCE_PACKAGE_GRACEFUL_EXCEPTION
+internal class GracefulException : Exception


Can we remove all the exception throwing? We already report diagnostics, right?
The calling code can convert diagnostics to an exception if needed.

I think it will be best to remove usage of this type from the source package. But, it will involve a decent chunk of refactoring.

tmat · 2025-10-22T16:56:29Z

src/Cli/Microsoft.DotNet.FileBasedPrograms/AppDirectiveHelpers.cs

+                // Also normalize blackslashes to forward slashes to ensure the directive works on all platforms.
+                var sourceDirectory = Path.GetDirectoryName(context.SourceFile.Path) ?? ".";
+                var resolvedProjectPath = Path.Combine(sourceDirectory, directiveText.Replace('\\', '/'));
+                if (Directory.Exists(resolvedProjectPath))


Seems like this may depend on the process current directory. Should we abstract that? Depending on process state makes it hard to test and reuse the code in different environments

Are you suggesting adding some way to pass in a base directory for resolution here? Perhaps using the compilation’s SourceReferenceResolver if it exists?

Perhaps pass additional defaultDirectory parameter or something similar.

RikkiGibson · 2025-10-23T21:32:57Z

When attempting to use the package in Roslyn, I get the following exception in the analyzer.

[System.Resources.MissingManifestResourceException thrown]	
[External Code]	
FileBasedProgramsResources.GetResourceString(string resourceKey, string defaultValue) Line 15	C#
FileBasedProgramsResources.CouldNotFindProjectOrDirectory.get() Line 19	C#
CSharpDirective.Project.Parse(CSharpDirective.ParseContext context) Line 492	C#
CSharpDirective.Parse(CSharpDirective.ParseContext context) Line 315	C#
AppDirectiveHelpers.FindLeadingDirectives(SourceFile sourceFile, SyntaxTriviaList triviaList, DiagnosticBag diagnostics, ImmutableArray<CSharpDirective>.Builder builder) Line 165	C#

It looks like the .resources file is not getting packed, I will need to figure out how to do that.

tree output in the source package:

F:\.nuget\packages\microsoft.dotnet.filebasedprograms\10.0.200-dev
│   .nupkg.metadata
│   Icon.png
│   microsoft.dotnet.filebasedprograms.10.0.200-dev.nupkg
│   microsoft.dotnet.filebasedprograms.10.0.200-dev.nupkg.sha512
│   microsoft.dotnet.filebasedprograms.nuspec
│
├───build
│       Microsoft.DotNet.FileBasedPrograms.targets
│
└───contentFiles
    └───cs
        ├───net9.0
        │   │   .editorconfig
        │   │   AppDirectiveHelpers.cs
        │   │   ExternalHelpers.cs
        │   │   MSBuildUtilities.cs
        │   │
        │   └───FileBasedProgramsResources.resx
        │           Microsoft.DotNet.FileBasedPrograms.FileBasedProgramsResources.cs
        │
        └───netstandard2.0
            │   .editorconfig
            │   AppDirectiveHelpers.cs
            │   ExternalHelpers.cs
            │   MSBuildUtilities.cs
            │
            └───FileBasedProgramsResources.resx
                    Microsoft.DotNet.FileBasedPrograms.FileBasedProgramsResources.cs

RikkiGibson · 2025-10-23T23:22:48Z

I am unsure whether it is better to include the outputs of compiling the .resx/.xlf files, and add some kind of targets to ensure the resulting .resources get embedded into the resource assemblies of the consuming project, or, to actually embed the original .resx/.xlf as items and have the consuming project compile them.

I didn't see any examples of a source-only package which includes resource files. Do we have any examples of that in our org? Tagging @tmat @baronfel in case you happen to know (or know where to look.)

tmat · 2025-10-24T02:57:33Z

I am unsure whether it is better to include the outputs of compiling the .resx/.xlf files, and add some kind of targets to ensure the resulting .resources get embedded into the resource assemblies of the consuming project, or, to actually embed the original .resx/.xlf as items and have the consuming project compile them.

I didn't see any examples of a source-only package which includes resource files. Do we have any examples of that in our org? Tagging @tmat @baronfel in case you happen to know (or know where to look.)

Resources in source packages are PITA. Is it possible to avoid?

RikkiGibson · 2025-10-24T03:17:58Z

The resources are basically the error messages for bad #:property etc directives. It would be possible to require the consumer of the source package to supply the resource strings, and copy paste them into the .resx of each repo essentially, but it seems unfortunate.

This is the content that would be copied (and possibly translations performed again, unless, perhaps with comments we give the translators a heads up that there is another copy elsewhere to use as a reference.)

sdk/src/Cli/Microsoft.DotNet.FileBasedPrograms/FileBasedProgramsResources.resx

Lines 120 to 171 in 839754f

    
           <data name="CouldNotFindAnyProjectInDirectory" xml:space="preserve"> 
        
             <value>Could not find any project in `{0}`.</value> 
        
           </data> 
        
           <data name="CouldNotFindProjectOrDirectory" xml:space="preserve"> 
        
             <value>Could not find project or directory `{0}`.</value> 
        
           </data> 
        
           <data name="MoreThanOneProjectInDirectory" xml:space="preserve"> 
        
             <value>Found more than one project in `{0}`. Specify which one to use.</value> 
        
           </data> 
        
           <data name="PropertyDirectiveInvalidName" xml:space="preserve"> 
        
             <value>Invalid property name: {0}</value> 
        
             <comment>{0} is an inner exception message.</comment> 
        
           </data> 
        
           <data name="PropertyDirectiveMissingParts" xml:space="preserve"> 
        
             <value>The property directive needs to have two parts separated by '=' like '#:property PropertyName=PropertyValue'.</value> 
        
             <comment>{Locked="#:property"}</comment> 
        
           </data> 
        
           <data name="StaticGraphRestoreNotSupported" xml:space="preserve"> 
        
             <value>Static graph restore is not supported for file-based apps. Remove the '#:property'.</value> 
        
             <comment>{Locked="#:property"}</comment> 
        
           </data> 
        
           <data name="DirectiveError" xml:space="preserve"> 
        
             <value>error</value> 
        
             <comment>Used when reporting directive errors like "file(location): error: message".</comment> 
        
           </data> 
        
           <data name="InvalidDirectiveName" xml:space="preserve"> 
        
             <value>The directive should contain a name without special characters and an optional value separated by '{1}' like '#:{0} Name{1}Value'.</value> 
        
             <comment>{0} is the directive type like 'package' or 'sdk'. {1} is the expected separator like '@' or '='.</comment> 
        
           </data> 
        
           <data name="CannotConvertDirective" xml:space="preserve"> 
        
             <value>Some directives cannot be converted. Run the file to see all compilation errors. Specify '--force' to convert anyway.</value> 
        
             <comment>{Locked="--force"}</comment> 
        
           </data> 
        
           <data name="DuplicateDirective" xml:space="preserve"> 
        
             <value>Duplicate directives are not supported: {0}</value> 
        
             <comment>{0} is the directive type and name.</comment> 
        
           </data> 
        
           <data name="QuoteInDirective" xml:space="preserve"> 
        
             <value>Directives currently cannot contain double quotes (").</value> 
        
           </data> 
        
           <data name="InvalidProjectDirective" xml:space="preserve"> 
        
             <value>The '#:project' directive is invalid: {0}</value> 
        
             <comment>{0} is the inner error message.</comment> 
        
           </data> 
        
           <data name="MissingDirectiveName" xml:space="preserve"> 
        
             <value>Missing name of '{0}'.</value> 
        
             <comment>{0} is the directive name like 'package' or 'sdk'.</comment> 
        
           </data> 
        
           <data name="UnrecognizedDirective" xml:space="preserve"> 
        
             <value>Unrecognized directive '{0}'.</value> 
        
             <comment>{0} is the directive name like 'package' or 'sdk'.</comment> 
        
           </data>

jjonescz · 2025-10-24T08:06:32Z

to actually embed the original .resx/.xlf as items and have the consuming project compile them.

I would do this and on the consuming side something like this could work?

<PackageReference Include="microsoft.dotnet.filebasedprograms" GeneratePathProperty="true" />

<EmbeddedResource Include="$(PkgMicrosoft_DotNet_FileBasedPrograms)contentfiles\cs\netstandard2.0\FileBasedProgramResources.resx" />

github-actions bot added the Area-Infrastructure label Oct 20, 2025

RikkiGibson mentioned this pull request Oct 20, 2025

Fbp live directive diagnostics dotnet/roslyn#80575

Draft

build-analysis bot mentioned this pull request Oct 21, 2025

Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. dotnet/dnceng#5922

Open

3 tasks

RikkiGibson commented Oct 21, 2025

View reviewed changes

build-analysis bot mentioned this pull request Oct 21, 2025

TemplateEngineToolDiscovery Network Failure: Unable to read data from the transport connection: Connection reset by peer #51154

Open

jjonescz reviewed Oct 22, 2025

View reviewed changes

tmat reviewed Oct 22, 2025

View reviewed changes

Add source package for file based programs support

28adee2

RikkiGibson changed the base branch from main to release/10.0.2xx October 23, 2025 19:03

RikkiGibson force-pushed the fbp-source-package branch from 17b3342 to 28adee2 Compare October 23, 2025 19:03

Put TextSpan back in SimpleDiagnostic.Position

d08c58f

This was referenced Oct 23, 2025

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

Template engine tests hitting known parallelism issue in GetVisualStudioInstances #44878

Open

fix formatting

4ce6414

cleanup

839754f


		namespace Microsoft.DotNet.FileBasedPrograms;

		internal static class AppDirectiveHelpers

Add source package for file-based programs support #51373

Are you sure you want to change the base?

Add source package for file-based programs support #51373

Conversation

RikkiGibson commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RikkiGibson Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmat Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RikkiGibson commented Oct 23, 2025

Uh oh!

RikkiGibson commented Oct 23, 2025

Uh oh!

tmat commented Oct 24, 2025

Uh oh!

RikkiGibson commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jjonescz commented Oct 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

RikkiGibson commented Oct 20, 2025 •

edited

Loading

RikkiGibson Oct 21, 2025 •

edited

Loading

tmat Oct 22, 2025 •

edited

Loading

RikkiGibson commented Oct 24, 2025 •

edited

Loading