Don't use chcp. #913

janlazo · 2019-11-29T04:35:14Z

User should fix their terminal font to display unicode characters.
Terminal programs can only use Wide String APIs.
For Vim, this requires +multi_byte feature and set encoding=utf-8
in the user's vimrc. iconv() and libcallnr to get the current codepage (can this be cached?).
Neovim always defaults to set encoding=utf-8.

vim-plug runs commands in a batchfile with utf-8 characters.
On terminal Vim, run the batchfile in a separate minimized console
to not break the terminal console.

https://dev.to/mattn/please-stop-hack-chcp-65001-27db

cc @mattn @junegunn

Close #908

janlazo · 2019-11-29T04:46:02Z

plug.vim

    \ + ['endlocal'],
    \ 'v:val."\r"')
  endfunction

  function! s:batchfile(cmd)
    let batchfile = tempname().'.bat'
    call writefile(s:wrap_cmds(a:cmd), batchfile)
-    let cmd = plug#shellescape(batchfile, {'shell': &shell, 'script': 1})
-    if &shell =~# 'powershell\.exe$'
+    let cmd = plug#shellescape(batchfile, {'shell': &shell, 'script': 0})


It should have been 0 to begin with because the batchfile will run on the shell, not inside another batchfile.

janlazo · 2019-11-29T04:46:55Z

plug.vim

    return s:shellesc_cmd(a:arg, script)
-  elseif shell =~# 'powershell\.exe$' || shell =~# 'pwsh$'
+  elseif shell =~# 'powershell\.exe' || shell =~# 'pwsh$'


$ was a premature optimization because 8.0.1455 and relatex 8.1.x patches were not considered.

janlazo · 2019-11-29T04:50:26Z

plug.vim

+  if !has('nvim')
+    \ && (has('win32') || has('win32unix'))
+    \ && (!has('multi-byte') || &encoding !=# 'utf-8')
+    return s:err('Only Vim with +multi_byte feature is supported. Add `set encoding=utf-8` in your vimrc before running `plug#begin()`.')


@mattn Is this a reasonable restriction to guarantee that the batchfile has utf-8 encoding?

Why do you check support of encoding=utf-8 ?

I want to guarantee that writefile works with unicode and batchfile has unicode encoding. I don't know enough about the Vim internals to know if this is needed or is insufficent so I'm asking. Maybe writefile with b flag is enough?

The batch file must be written in code page NOT unicode. So you can use iconv().

let cp = libcallnr('kernel32.dll', 'GetACP', 0) let text = iconv(text, &encoding, printf('cp%d', cp))

iconv replaces unicode characters with ?. I'd rather use powershell if the batchfile cannot be executed with unicode characters and non-unicode codepage.

Why you have to handle unicode characters?

plug.vim

mattn · 2019-11-29T05:41:55Z

This is my suggestion.

https://gist.github.com/mattn/9182880f9080d728338bfea9ef1edccd

plug.vim

janlazo · 2019-11-30T00:04:55Z

Batchfile is required on Vim to support multiple shells while using cmd.exe syntax and not changing the user's shell settings, which depend on the editor and OS. I will reconsider if #594 is finished before removing the batchfile code because Vim's job_start does not use the user's shell.

On Neovim, only :! is problematic. I'll probably use terminal if it can be blocked.

mattn · 2019-12-02T06:43:47Z

plug.vim

@@ -139,6 +139,11 @@ function! s:define_commands()
  if !executable('git')
    return s:err('`git` executable not found. Most commands will not be available. To suppress this message, prepend `silent!` to `call plug#begin(...)`.')
  endif
+  if !has('nvim')
+    \ && (has('win32') || has('win32unix'))
+    \ && (!has('multi-byte') || !has('iconv'))


s/multi-byte/multi_byte/

mattn · 2019-12-02T13:57:25Z

LGTM

plug.vim

Changing chcp breaks cmd.exe if switching from multi-byte to 65001. cmd.exe depends on codepage to parse batchfile so batchfile cannot have unicode characters. Target powershell if unicode support is required. User should fix their terminal font to display unicode characters. Terminal programs can only use Wide String APIs. For Vim, this requires +multi_byte feature and `set encoding=utf-8` in the user's vimrc. Neovim always defaults to `set encoding=utf-8`. https://dev.to/mattn/please-stop-hack-chcp-65001-27db

janlazo · 2019-12-11T13:27:48Z

Works with my config on a Windows VM. Tested with cmd.exe, powershell.exe, and bash.exe. Tested only with codepage 1252 (English).

janlazo · 2019-12-11T13:29:46Z

@mattn Thanks for the help.

Backport junegunn/vim-plug#913

janlazo · 2019-12-15T23:50:38Z

@junegunn junegunn/fzf@f68017d, port of this PR, breaks filtering for Commands. Back to square one 😞

De-quote only if first and last character of &shell are double-quotes. Avoid string escape issues when passing escaped '&shell' values.

janlazo commented Nov 29, 2019

View reviewed changes

mattn reviewed Nov 29, 2019

View reviewed changes

plug.vim Outdated Show resolved Hide resolved

mattn reviewed Nov 29, 2019

View reviewed changes

plug.vim Outdated Show resolved Hide resolved

janlazo force-pushed the batchfile-unicode branch from aec464e to c86659e Compare November 30, 2019 00:15

janlazo changed the title ~~Don't use chcp. Spawn console for utf-8 batchfile~~ Don't use chcp. Nov 30, 2019

janlazo force-pushed the batchfile-unicode branch from 73407c6 to 2de5b4a Compare December 2, 2019 06:20

mattn reviewed Dec 2, 2019

View reviewed changes

mattn approved these changes Dec 2, 2019

View reviewed changes

junegunn reviewed Dec 5, 2019

View reviewed changes

plug.vim Show resolved Hide resolved

janlazo mentioned this pull request Dec 7, 2019

Add MSYS2 support as a vim plugin junegunn/fzf#1677

Merged

janlazo force-pushed the batchfile-unicode branch from b4187cd to ef84ae5 Compare December 8, 2019 02:39

janlazo force-pushed the batchfile-unicode branch from ef84ae5 to 4f18713 Compare December 11, 2019 11:49

janlazo merged commit 359ce90 into junegunn:master Dec 11, 2019

janlazo deleted the batchfile-unicode branch December 11, 2019 13:29

junegunn added a commit to junegunn/fzf that referenced this pull request Dec 12, 2019

[windows/vim] Encode batchfile in current codepage

f68017d

Backport junegunn/vim-plug#913

DervishD mentioned this pull request Dec 26, 2019

vim-plug requires +iconv but doesn't work with +iconv/dyn #920

Closed

8 tasks

janlazo referenced this pull request in janlazo/dotvim8 Oct 16, 2021

vim: de-quote 'shell' option

95f56e3

De-quote only if first and last character of &shell are double-quotes. Avoid string escape issues when passing escaped '&shell' values.

Uh oh!

Don't use chcp. #913

Don't use chcp. #913

Uh oh!

Conversation

janlazo commented Nov 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mattn commented Nov 29, 2019

Uh oh!

Uh oh!

janlazo commented Nov 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattn commented Dec 2, 2019

Uh oh!

Uh oh!

janlazo commented Dec 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

janlazo commented Dec 11, 2019

Uh oh!

janlazo commented Dec 15, 2019

Uh oh!

Uh oh!

janlazo commented Nov 29, 2019 •

edited

Loading

janlazo commented Nov 30, 2019 •

edited

Loading

janlazo commented Dec 11, 2019 •

edited

Loading