Skip to content

Proposal: scriptstrict directive (Do Not Merge)#3857

Closed
mattn wants to merge 3 commits intovim:masterfrom
mattn:scriptstrict
Closed

Proposal: scriptstrict directive (Do Not Merge)#3857
mattn wants to merge 3 commits intovim:masterfrom
mattn:scriptstrict

Conversation

@mattn
Copy link
Member

@mattn mattn commented Jan 24, 2019

I propose to add new directive command scriptstrict to indicate that the code should be executed in "strict mode".

Background

In the specification of Vim script, Vim script allow to put . for two expression.

  1. concating two string values
  2. reference item of dictionary

code1

let a = 'a'
let b = 'b'
echo a.b

code2

let a = {'b': 'c'}
echo a.b

S expression (pseudo) of code1 is:

(progn
  (let = a 'a')
  (let = b 'b')
  (echo (concat a b))
)

And code2 is:

(progn
  (let a (dict ('b' 'c'))
  (echo (gethash 'b' a))
)

So Vim script is not possible to generate AST (Abstruct Syntax Tree) since a.b can not be parsed to the node of tree. To make Vim script faster, we should compile Vim script to AST, and make AST walker. However, the meaning of . operator is indefinite until the type of left-side express is recognized.

Proposal

I suggest that the operator for the concat strings using a.b is obsolete. However, Vim place too much compatibilities. So I added scriptstrict directive. This works as Ex command like scriptencoding. When put scriptstrict in your Vim script, Vim does not allow expression a.b to be interpret as concat strings. If you want to do it, the expression must be a .b or a . b. This pull-request is not ready to merge since I thought we have to discuss. But if you build this pull-request, Following Vim script works as well.

" Concating strings: This is OK since not strict mode
let a = 'a'
let b = 'b'
echo a.b

" Reference item in dictionary: This is OK since not strict mode
let a = {'b': 'b'}
echo a.b

" Then, scriptstrict flag on.
scriptstrict

" Concating strings: This is NG since strict mode
let a = 'a'
let b = 'b'
try
  echo a.b
catch
  echohl ErrorMsg | echo 'Do not use "." for concating strings!' | echohl None
endtry

" Concating strings but include space before dot: This is OK
let a = 'a'
let b = 'b'
echo a .b

" Reference item in dictionary: This is OK even when strict mode
let a = {'b': 'b'}
echo a.b

Please note that I don't say that Vim script is possible to implement AST parser if the a.b will be removed. This is one of candidate that have to be removed.

I would like to hear your opinions. Thanks.

@codecov-io
Copy link

Codecov Report

Merging #3857 into master will decrease coverage by 0.01%.
The diff coverage is 30.76%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3857      +/-   ##
==========================================
- Coverage   78.66%   78.64%   -0.02%     
==========================================
  Files         103      103              
  Lines      141899   141912      +13     
==========================================
- Hits       111618   111608      -10     
- Misses      30281    30304      +23
Impacted Files Coverage Δ
src/ex_docmd.c 80.34% <ø> (ø) ⬆️
src/ex_cmds2.c 84.78% <0%> (-0.2%) ⬇️
src/buffer.c 79.84% <100%> (ø) ⬆️
src/option.c 85.77% <100%> (ø) ⬆️
src/eval.c 85.07% <25%> (-0.06%) ⬇️
src/main.c 65.25% <50%> (-0.03%) ⬇️
src/if_xcmdsrv.c 83.48% <0%> (-0.54%) ⬇️
src/gui_gtk_x11.c 48.37% <0%> (-0.3%) ⬇️
src/gui.c 58% <0%> (-0.11%) ⬇️
src/ex_cmds.c 79.99% <0%> (-0.1%) ⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4456ab5...9f8fe1e. Read the comment docs.

@brammool
Copy link
Contributor

brammool commented Jan 24, 2019 via email

@mattn
Copy link
Member Author

mattn commented Jan 24, 2019

Yes, I also thought that scriptstrict should take an argument for the script version. And the version is used for checking version what Vim improved. And if the version is same as Vim's version, Vim compile the script to AST.
I don't worry about the '3' + 4 since the syntax tree is always fixed to (+ '3' 4) even if type of results are different for each them. Just return which one by + operator. The problem is that . operator have indeterminate meanings:

let a = "a"
let b = "b"
let c = "c"
let x = a.b.c

This mean:

(let = x (concat a b c))

But

let a = {"b": {"c"}}
let x = a.b.c

This mean:

(let = x (gethash 'c' (gethash 'b' a))

@chdiza
Copy link

chdiza commented Jan 24, 2019

I don't see what's wrong with "." for string concatenation (still less why it would be "obsolete"). It seems to me that it's the dictionary notation that should be changed.

@mattn
Copy link
Member Author

mattn commented Jan 24, 2019

No. Vim script have dictionary function. And many plugins have the codes.

@chdiza
Copy link

chdiza commented Jan 25, 2019

Vim script also has a string concatenation operator. And many plugins and scripts use it.

@mattn
Copy link
Member Author

mattn commented Jan 25, 2019

What do you worry about? I'm not trying to completely delete string concatings. If you want to use string concatings with "a.b", you can use if you don't put "scriptversion". This is only thing that scripts are not optimized in future.

@chdiza
Copy link

chdiza commented Jan 25, 2019

I worry that no reason has been given for taking "." away from string concatenation (in "strict mode") instead of taking "." away from dictionary expressions. Why not just require (in "strict mode") the dictionary expression a.b to instead be a[b]?

@inkarkat
Copy link
Contributor

@chdiza Dictionary expressions have always been written as a.b, so by changing that, all code has to be adapted. Whereas with string concatenation, the variant with whitespace around . has been used more often (at least that's my gut feeling) than the variant without, and (you might have missed that, as it's easy to overlook), the suggestion here is to only remove the latter variant. Code that uses a . b (or the inconsistent a .b) for concatenating could remain as-is. Also, the necessary change would be to just add missing whitespace, not changing the syntax altogether, like in your counter-proposal.


I concur with Bram that such endeavor should be part of a larger redesign. If this is done piecemeal through many minor revisions, and without discernible (performance) benefits, I doubt this will get much traction. Why should a plugin writer make changes that break backwards compatibility if there's a chance this soon needs to be adapted again?

Also, this probably won't silence those critics that dislike Vimscript in general (unless this gets a complete, radical overhaul, and then shouldn't be called Vimscript any longer). I wonder if we couldn't just leverage any performance optimizations from other scripting languages: If, say, Python does a great job there already, why not just invest some time in better and efficient Vim bindings, and then tell plugin writers to implement large plugins in Python?

@chdiza
Copy link

chdiza commented Jan 25, 2019

Dictionary expressions have always been written as a.b

Haven't they also always been written as a[b]? Haven't string concats always been writable as a.b?

@chdiza
Copy link

chdiza commented Jan 25, 2019

Dictionary expression certainly can be written a[b] (whether or not it was always thus), and often is so written; so it just isn't true that all dictionary code would have to be changed.

@inkarkat
Copy link
Contributor

@chdiza In a.b, b is a a literal key (this works only for keys that consist entirely of letters, digits and underscore, as per :help E713). If you use a[b], b is a Vimscript expression (variable). To be equivalent, it would have to be written as a['b'] or a["b"].

@brammool
Copy link
Contributor

brammool commented Jan 25, 2019 via email

@chdiza
Copy link

chdiza commented Jan 25, 2019

Thus it's best if Vim does something that users expect from other languages.

Vim will need to keep "a[b]" anyway, because otherwise there's no way to distinguish between "a.b" where a is a dictionary and "b" is one of its string keys, and "a.b" where a is a dictionary and b is a variable whose value is one of a's keys.

let a = {"b" : 1, "c": 2, "foo": 3}
let b = "foo"
echo a.b

That will produce 1, so to get 3 via the variable b one will need to do echo a[b]. The latter is allegedly not in line with the expectations of users of (certain) other languages. So the battle to stay in line with those expectations was lost long ago.

@mattn
Copy link
Member Author

mattn commented Jan 26, 2019

As mentioned in above, Vim script have dictionary function.

let a = {"value": 1}
function! a.b(k)
  echo self[a:k]
endfunction

call a.b("value")

Many Vim plugins are using this idiom.

ex: https://github.com/vim-jp/vital.vim

@mattn
Copy link
Member Author

mattn commented Jan 26, 2019

And many script author is using spaces arround . for concat strings habitually to avoid consufing.

@mattn
Copy link
Member Author

mattn commented Jan 26, 2019

I want to suggest one more thing to be avoided confusing.

function! s:test()
  let l:l = 1
  let l:k = 2
  let l:c = [1,2,3]
  echo l:c[l:l:k]
endfunction

call s:test()

This is parsed as:

echo l:c[l:l: k]

But can be confused to:

echo l:c[l :l:k]

In strict mode, spaces arround : must be included.

echo l:c[l:l : k]

@Houl
Copy link
Contributor

Houl commented Jan 26, 2019

Any reason to leave out a. b for string concatenation? This is what I use most often, rather than a . b. I've never seen anybody using a .b (maybe I should look at more code ...). Reserving a.b for dict access should be fine. Adding another operator .. for string concatenation should be fine also (then whitespace wouldn't matter).

@mattn
Copy link
Member Author

mattn commented Jan 26, 2019

We don't mean that we will remove a.b for string concatenation. Just small thing that will not be optimized.

@andymass
Copy link

andymass commented Jan 26, 2019

l:c[l:l:k] is confusing to but it's not a true ambiguity in the grammar, in the sense that the parsing does not depend on the type of the variables. I would think l:c[l:(l:k)] could be allowed.

@Houl is just saying to allow a. b under "strict," so a space on either side is allowable and means string concatenation.

By the way, can such a directive (or a different name) have the effect of:

let s:save_cpo = &cpo
set cpo&vim
...
let &cpo = s:save_cpo

@brammool brammool closed this in 558ca4a Apr 4, 2019
seandewar added a commit to seandewar/neovim that referenced this pull request May 21, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a
seandewar added a commit to seandewar/neovim that referenced this pull request May 21, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513 and
vim/vim@68e6560.
seandewar added a commit to seandewar/neovim that referenced this pull request May 21, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513,
vim/vim@68e6560 and
vim/vim@911ead1.
seandewar added a commit to seandewar/neovim that referenced this pull request May 22, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513,
vim/vim@68e6560 and
vim/vim@911ead1.

Include latest repeat.txt changes for :scriptversion from
vim/vim@62e1bb4.
seandewar added a commit to seandewar/neovim that referenced this pull request Jun 2, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513,
vim/vim@68e6560 and
vim/vim@911ead1.

Include latest repeat.txt changes for :scriptversion from
vim/vim@62e1bb4.
seandewar added a commit to seandewar/neovim that referenced this pull request Jun 5, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513,
vim/vim@68e6560 and
vim/vim@911ead1.

Include latest repeat.txt changes for :scriptversion from
vim/vim@62e1bb4.
seandewar added a commit to seandewar/neovim that referenced this pull request Jun 25, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513,
vim/vim@68e6560 and
vim/vim@911ead1.

Include latest repeat.txt changes for :scriptversion from
vim/vim@62e1bb4.
seandewar added a commit to seandewar/neovim that referenced this pull request Jun 26, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513,
vim/vim@68e6560 and
vim/vim@911ead1.

Include latest repeat.txt changes for :scriptversion from
vim/vim@62e1bb4.
seandewar added a commit to seandewar/neovim that referenced this pull request Aug 26, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513,
vim/vim@68e6560 and
vim/vim@911ead1.

Include latest repeat.txt changes for :scriptversion from
vim/vim@62e1bb4.
seandewar added a commit to seandewar/neovim that referenced this pull request Aug 26, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

Include latest eval.txt changes for scriptversion-2 from v8.1.1513,
vim/vim@68e6560 and
vim/vim@911ead1.

Include latest repeat.txt changes for :scriptversion from
vim/vim@62e1bb4.
seandewar added a commit to seandewar/neovim that referenced this pull request Sep 10, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

:scriptversion is N/A, but ":let ..=" is relevant.

N/A patches for version.c

vim-patch:8.1.1188: not all Vim variables require the v: prefix

Problem:    Not all Vim variables require the v: prefix.
Solution:   When scriptversion is 3 all Vim variables can only be used with
            the v: prefix.  (Ken Takata, closes vim/vim#4274)
vim/vim@d2e716e

vim-patch:8.1.1190: has('vimscript-3') does not work

Problem:    has('vimscript-3') does not work.
Solution:   Add "vimscript-3" to the list of features.
vim/vim@93a4879

vim-patch:8.1.2038: has('vimscript-4') is always 0

Problem:    has('vimscript-4') is always 0.
Solution:   Add "vimscript-4" to the feature table. (Naruhiko Nishino,
            closes vim/vim#4941)
vim/vim@af91438

vim-patch:8.1.2043: not sufficient testing for quoted numbers

Problem:    Not sufficient testing for quoted numbers.
Solution:   Add a few more test cases.
vim/vim@ea8dcf8
seandewar added a commit to seandewar/neovim that referenced this pull request Sep 11, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

:scriptversion is N/A, but ":let ..=" is relevant.

N/A patches for version.c

vim-patch:8.1.1188: not all Vim variables require the v: prefix

Problem:    Not all Vim variables require the v: prefix.
Solution:   When scriptversion is 3 all Vim variables can only be used with
            the v: prefix.  (Ken Takata, closes vim/vim#4274)
vim/vim@d2e716e

vim-patch:8.1.1190: has('vimscript-3') does not work

Problem:    has('vimscript-3') does not work.
Solution:   Add "vimscript-3" to the list of features.
vim/vim@93a4879

vim-patch:8.1.2038: has('vimscript-4') is always 0

Problem:    has('vimscript-4') is always 0.
Solution:   Add "vimscript-4" to the feature table. (Naruhiko Nishino,
            closes vim/vim#4941)
vim/vim@af91438
lewis6991 pushed a commit to lewis6991/neovim that referenced this pull request Dec 12, 2021
Problem:    Cannot enforce a Vim script style.
Solution:   Add the :scriptversion command. (closes vim/vim#3857)
vim/vim@558ca4a

:scriptversion is N/A, but ":let ..=" is relevant.

N/A patches for version.c

vim-patch:8.1.1188: not all Vim variables require the v: prefix

Problem:    Not all Vim variables require the v: prefix.
Solution:   When scriptversion is 3 all Vim variables can only be used with
            the v: prefix.  (Ken Takata, closes vim/vim#4274)
vim/vim@d2e716e

vim-patch:8.1.1190: has('vimscript-3') does not work

Problem:    has('vimscript-3') does not work.
Solution:   Add "vimscript-3" to the list of features.
vim/vim@93a4879

vim-patch:8.1.2038: has('vimscript-4') is always 0

Problem:    has('vimscript-4') is always 0.
Solution:   Add "vimscript-4" to the feature table. (Naruhiko Nishino,
            closes vim/vim#4941)
vim/vim@af91438
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants