On Windows 10, at launch of LLDB, program is unable to access stdout or stdin

[KimTheLearner]

Issue Description

On Windows 10, at the launch of Swift's custom LLDB by the Swift extension for VS Code (vscode-swift), the user program's output from the 'print()' statement is nowhere to be seen, and the attempt to do 'readline()' gets an EOF result.

The issue seems most likely to be in the Swift extension for VS Code, or possibly in the CodeLLDB extension. This is based on the 'Variations' section, below.

This is in the global settings.json file, which correctly points to the Swift custom LLDB:

{
    "lldb.library": "C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\bin\\liblldb.dll"
}

Discovered during work on #301.

Let me know if you need me to provide any additional information.

Environment

OS: Windows 10
VSCode version: 1.67.1
Swift extension version: 0.5.2
CodeLLDB version: 1.7.0
Compiler: Swift 5/04/22 'latest snapshot of main': swift-DEVELOPMENT-SNAPSHOT-2022-05-04-a-windows10.exe

• Note: The results below differ substantially if the slightly earlier release, Swift 5.6.1, is used - see On Windows 10, Launch of LLDB gets error regarding init_fs_encoding #301.

Test Case

md hello
cd hello
swift package init --type=executable
place these lines into main.swift:

import Foundation

print("Hello, world!")
print("enter something  ->", terminator: "")

guard let answer = readLine() else {
    print("End of File was reached - aborting...")
    exit(1)
}
print("readline() did not see EOF")

open folder with VS Code
compile with: Terminal > Build Debug hello
hello.exe is built.
set a breakpoint on 'Hello,world' line.
click Run > Start Debugging, and then step through the program.

Expected Behavior

This expectation is based on one of the 'Variations' - see below. At this point you should see the following in VS Code's lower pane > Terminal tab. When the prompt "enter something" appears, you should be able to enter text, for example "abc". Control should skip over the 'else' to the print statement at the bottom of the program. At the end of execution, the Terminal tab should show:

Hello, world!
enter something  ->abc
readline() did not see EOF

Actual Behavior

A new command prompt window is created, titled with 'hello.exe'. It remains empty throughout program execution.

The output of the print statements is nowhere to be seen - neither in the VS Code lower pane in tab 'Terminal', nor in the new empty command prompt window.

As you step over the 'readline()' statement, the program fails to wait for you to enter text, control simply falls into the 'else' part of the guard, where EOF is handled.

This output appears in the lower pane, tab 'Output', channel LLDB:

configuration: {
  type: 'lldb',
  request: 'launch',
  name: 'Debug hello',
  program: '${workspaceFolder:hello}/.build/debug/hello',
  args: [],
  cwd: '${workspaceFolder:hello}',
  preLaunchTask: 'swift: Build Debug hello',
  __configurationTarget: 5,
  relativePathBase: 'k:\\Kim Shared\\CodeLab\\SwiftTests\\hello'
}
[adapter\src\terminal.rs:104] FreeConsole() = 1
[adapter\src\terminal.rs:105] AttachConsole(pid) = 1
[adapter\src\terminal.rs:109] FreeConsole() = 1
[2022-05-17T23:49:11.082Z ERROR codelldb::debug_session] Internal debugger error: Invalid frame reference: 1001

There seems to be some variability in this error log - in another similar case, shown in #301, in the comments by me starting with "Alrighty", I had this instead:

[adapter\src\terminal.rs:104] FreeConsole() = 1
[adapter\src\terminal.rs:105] AttachConsole(pid) = 1
[adapter\src\terminal.rs:109] FreeConsole() = 1
ERROR(Python) 20:38:54 codelldb: Traceback (most recent call last):
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 152, in evaluate
    value = evaluate_in_context(pycode, is_simple_expr, context)
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 288, in evaluate_in_context
    return eval(code, eval_globals, eval_locals)
  File "<input>", line 1, in <module>
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 269, in __missing__
    raise VariableNotFound(name)
codelldb.VariableNotFound: Variable 'vars' not found

[2022-05-12T01:38:54.750Z ERROR codelldb::debug_session] Variable 'vars' not found
Debug adapter exit code=0, signal=null.

Variations

When I comment out the "lldb.library" entry in settings.json, thus forcing the standard LLDB (poorly named, I meant the LLDB packaged with the CodeLLDB extension) to be used instead of the Swift custom LLDB, then do Run > Start Debugging, the print() statement output appears in the lower pane of VS Code, in the Terminal tab; likewise, for readline(), the user enters text in the same Terminal tab. That feels like the correct behavior to me, for VSCode-based debugging. This is the basis of my expected behavior, above.

When I run the Swift LLDB from the command prompt, the program uses stdin/stdout just fine, and that occurs in a command prompt window that lldb launches for the program.

Likewise when I run the program directly from the command prompt (no LLDB), the prog uses stdin/stdout just fine in that same command prompt window.

Taking all these variations together, these indicate that the source of the problem must be in the setup done in the Swift extension as it prepares to invoke the LLDB debug adapter, or possibly the debug adapter is mishandling something when the lldb.library is set. If no objections, we propose that we begin the investigation in the Swift extension (see #301 at the bottom).

[KimTheLearner]

argg, @adam-fowler , I just saw your comment of yesterday. I did check for any follow-up from you or @compnerd on #301 just before creating this issue 309, but alas, I was looking at a stale view.

So, for good or bad, I've already created this issue, so I am going to paste your reply here, and then let's continue any discussion of this issue here, in 309.

@compnerd I've had a quick look at this and sorry to pass the buck but I think this is more a Swift lldb issue. Running from the command line the standard lldb prints directly to the command line any output from the application. If you use the swift lldb it opens up a new window for the output. I'm assuming this difference is what is causing the issue inside vscode-swift as there is no output to stdout/stderr it is all redirected to the window that opens.

[compnerd]

Hmm, I don't see how lldb would create a new console for the application, it does a standard process creation AFAIK. What is standard lldb here for context? Is it the lldb from llvm.org? Built at what revision?

[KimTheLearner]

I am trying to reproduce Adam's case:

the standard lldb prints directly to the command line any output from the application

but I get different results. when I put the standard lldb foremost in my PATH, I get:

--> lldb hello

Fatal Python error: init_sys_streams: can't initialize sys standard streams
Python runtime state: core initialized
Traceback (most recent call last):
  File "C:\Program Files\Python\Python310\Lib\io.py", line 54, in <module>
ImportError: cannot import name 'text_encoding' from 'io' (unknown location)

my standard lldb is installed in:

c:\Users\Kim\.vscode\extensions\vadimcn.vscode-lldb-1.7.0\lldb\bin

and that is the same lldb.exe that Run > Start Debugging calls (per Process Explorer). This was installed by the LLDB extension for VS Code. That lldb\bin dir also contains its own copies of these python files: python3.dll and python39.dll. These are pulling io.py from my python 3.10 install (even when I remove that from the PATH). I won't be able to shed any light on standard lldb's behavior on my machine unless I can get it to ignore python310 folders.

[KimTheLearner]

also, Adam said

If you use the swift lldb it opens up a new window for the output. I'm assuming this difference is what is causing the issue inside vscode-swift as there is no output to stdout/stderr it is all redirected to the window that opens.

but that differs from what I see - no output is directed to the newly opened window.

[KimTheLearner]

A thought: since commenting out the "lldb.library" entry in settings.json makes things work as expected (stdout and stdin are in VS Code's 'Terminal' pane), could we assume that whoever is processing that setting is messing up the redirecting of stdout and stdin to the Terminal pane? And would that be some component of the LLDB extension?

[adam-fowler]

What is standard lldb here for context? Is it the lldb from llvm.org? Built at what revision?

When I say standard LLDB I mean the one that is packaged with vscode-lldb. Looking at the commit comments from vscode-lldb this would be from https://github.com/llvm/llvm-project, build id 1927 (not totally sure what build id indicates though).

but that differs from what I see - no output is directed to the newly opened window.

Sorry you are right when run via vscode-lldb no output is display to the window.

but I get different results. when I put the standard lldb foremost in my PATH

You need to resolve your python issues before you can compare the results.

As I said above the two versions of lldb when run from the command line react differently. We need to resolve this before looking at vscode-swift/lldb.

[compnerd]

Thinking about this a bit: the new window might be the result of AllocConsole. There is nothing that can be done about that - it is how console applications work on windows. https://docs.microsoft.com/en-us/windows/console/allocconsole

I think that we should look at how the inferior is launched. I suspect that this might be a bug in lldb-server. One quick test would be to try a “remote” (it can be run on the same host) debug session with ds2 (https://github.com/compnerd/ds2).

[KimTheLearner]

the new window might be the result of AllocConsole

yes, agreed, and I believe that is a valid technique to achieve clarity, i.e., when lldb is run as a CL tool, it would want the debugger commands and cmd outputs to be segregated from the program's interactions with the user, so 2 consoles is a good way to accomplish that.

You need to resolve your python issues before you can compare the results.

yes, agreed. I had updated my notes, perhaps after you read the initial notes, to indicate the same. I have a couple ideas and will report back.

[KimTheLearner]

I was able to get the standard LLDB to ignore my python310 folders, by doing:

rem PYTHONHOME=C:\Program Files\Python\Python310
rem PYTHONPATH=C:\Program Files\Python\Python310\Lib;C:\Program Files\Python\Python310\Lib\site-packages;
set PYTHONHOME=c:\Users\Kim\.vscode\extensions\vadimcn.vscode-lldb-1.7.0
set PYTHONPATH=c:\Users\Kim\.vscode\extensions\vadimcn.vscode-lldb-1.7.0\Lib;c:\Users\Kim\.vscode\extensions\vadimcn.vscode-lldb-1.7.0\Lib\site-packages;

This is in addition to the previous change to add the standard lldb foremost in my PATH, so that the one under C:\Library is not seen. With all that, the standard LLDB had the identical behavior described by Adam:

Running from the command line the standard lldb prints directly to the command line any output from the application.

I.e., the program output and input are intermingled with the LLDB command output and input:

    frame #0: 0x00007ff6146c127c hello.exe`main at main.swift:10:7
   7    
   8    
   9   
-> 10   print("Hello, world!")
   11
   12
   13   
(lldb) n
(lldb) Hello, world!                                                          ####### output from program
Process 7348 stopped                                                       ####### output from LLDB
* thread #1, stop reason = step over
    frame #0: 0x00007ff6146c1364 hello.exe`main at main.swift:15:7
   12
   13  
   14
-> 15   print("enter something  ->", terminator: "")
   16
   17   guard let answer = readLineaa() else {
   18       // EOF should NOT happen from std input.  sth is terribly wrong.
(lldb) n
enter something  ->Process 7348 stopped                      ####### output from program;  output from LLDB
* thread #1, stop reason = step over
    frame #0: 0x00007ff6146c146e hello.exe`main at main.swift:17:28
   14
   15   print("enter something  ->", terminator: "")
   16
-> 17   guard let answer = readLineaa() else {
   18       // EOF should NOT happen from std input.  sth is terribly wrong.
   19       print("\n\n\nEnd of File was reached - aborting...")
   20       exit(1)
(lldb) n                                                                          ####### user input to LLDB
(lldb) aaabbbccc                                                              ####### user input to program
Process 7348 stopped
* thread #1, stop reason = step over
    frame #0: 0x00007ff6146c15b8 hello.exe`main at main.swift:23:7
   20       exit(1)
   21   }
   22
-> 23   print("readline() did not see EOF")
   24
   25
   26   /*
(lldb) n                                                                    ####### output from LLDB
readline() did not see EOF                                                ####### output from program
Process 7348 stopped                                                  ####### output from LLDB 

So, clearly the Swift custom LLDB was changed to enable 2 separate consoles, one for LLDB and one for the program. I hope this helps.

Possibly that may have to be undone in order to get Swift custom LLDB working under VS Code. However, my own 2 cents would be that the optimal solution would be to fix the stdout/stdin redirecting while retaining the 2 separate consoles. Otherwise we are left with choosing between (A) cmd line LLDB working great with 2 consoles and VS Code-based debugging not working, or (B) the latter working and the former being degraded somewhat to 1 intermingled console. Thanks for considering.

[KimTheLearner]

I think that we should look at how the inferior is launched. I suspect that this might be a bug in lldb-server. One quick test would be to try a “remote” (it can be run on the same host) debug session with ds2 (https://github.com/compnerd/ds2).

I will try this next.

[KimTheLearner]

@compnerd, I see on your ds2 page that I do not have any of the many build tools listed there, and I would really prefer not to install a bunch of things and then uninstall them later, because of all the remnants that will leave.

Might you have a build that I could download and run on my Windows 10 machine?
Or, might someone else be able to attempt this?

Thanks.

[compnerd]

The only bits that you should need to install is Visual Studio; Visual Studio already provides CMake + Ninja. The only other dependency is flex and bison which is part of WinFixBison (which is why it is referenced), which doesn't need any installation.

That said ... https://github.com/compnerd/ds2/suites/5881671150/artifacts/198772803 or https://github.com/compnerd/ds2/suites/5881671150/artifacts/198772802 from a recent CI run should foot the bill