Python3 Windows and Winlogbeat updates#15831
Merged
andrewkroh merged 3 commits intoelastic:feature/python3from Jan 27, 2020
Merged
Python3 Windows and Winlogbeat updates#15831andrewkroh merged 3 commits intoelastic:feature/python3from
andrewkroh merged 3 commits intoelastic:feature/python3from
Conversation
This updates the Vagrantfile and addresses a few issues. I installed Python3 on Windows
via chocolotey and the python MSI and neither provided a `python3` command so I adjusted
the build tooling to use `python` on Windows.
Another issue I hit was that under Vagrant with the shared filesystem, `pip` would not
install into the virtual environment. It only will install when moving the PYTHON_ENV
off of the shared filesystem. So when provisining the Vagrant VM it will now put
PYTHON_ENV in C:\beats-python-env. Remember this if you need need to do a completely
clean build on Windows+Vagrant.
There were some encoding issues I found while reading output files containing unicode.
So I modified the libbeat's beat.py to specify `utf_8` encoding when opening output files.
I was seeing errors like:
```
======================================================================
ERROR: eventlogging - UTF-16 characters
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Gopath\src\github.com\elastic\beats\winlogbeat\tests\system\test_eventlogging.py", line 187, in test_utf16_characters
evts = self.read_events(config={
File "C:\Gopath\src\github.com\elastic\beats\winlogbeat\tests\system\winlogbeat.py", line 110, in read_events
return self.read_output()
File "C:\Gopath\src\github.com\elastic\beats\winlogbeat\tests\system\../../../libbeat/tests/system\beat\beat.py", line 260, in read_output
for line in f:
File "C:\Python38\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 277: character maps to <undefined>
```
I bumped the Pillow version in the pip requirements because the newer version provides a
precompiled wheel for Python 3 on Windows. This avoids having to compiling it. I bumped
the Docker versions while debugging and just left them at the latest version since nothing broke.
I tested Winlogbeat and Metricbeat's `mage pythonUnitTest` on Windows 2019 and all tests passed.
63 tasks
Member
Author
|
Needs formatted.
|
jsoriano
approved these changes
Jan 27, 2020
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
* Python3 Windows and Winlogbeat updates
This updates the Vagrantfile and addresses a few issues. I installed Python3 on Windows
via chocolotey and the python MSI and neither provided a `python3` command so I adjusted
the build tooling to use `python` on Windows.
Another issue I hit was that under Vagrant with the shared filesystem, `pip` would not
install into the virtual environment. It only will install when moving the PYTHON_ENV
off of the shared filesystem. So when provisining the Vagrant VM it will now put
PYTHON_ENV in C:\beats-python-env. Remember this if you need need to do a completely
clean build on Windows+Vagrant.
There were some encoding issues I found while reading output files containing unicode.
So I modified the libbeat's beat.py to specify `utf_8` encoding when opening output files.
I was seeing errors like:
```
======================================================================
ERROR: eventlogging - UTF-16 characters
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Gopath\src\github.com\elastic\beats\winlogbeat\tests\system\test_eventlogging.py", line 187, in test_utf16_characters
evts = self.read_events(config={
File "C:\Gopath\src\github.com\elastic\beats\winlogbeat\tests\system\winlogbeat.py", line 110, in read_events
return self.read_output()
File "C:\Gopath\src\github.com\elastic\beats\winlogbeat\tests\system\../../../libbeat/tests/system\beat\beat.py", line 260, in read_output
for line in f:
File "C:\Python38\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 277: character maps to <undefined>
```
I bumped the Pillow version in the pip requirements because the newer version provides a
precompiled wheel for Python 3 on Windows. This avoids having to compiling it. I bumped
the Docker versions while debugging and just left them at the latest version since nothing broke.
I tested Winlogbeat and Metricbeat's `mage pythonUnitTest` on Windows 2019 and all tests passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This updates the Vagrantfile and addresses a few issues. I installed Python3 on Windows
via chocolotey and the python MSI and neither provided a
python3command so I adjustedthe build tooling to use
pythonon Windows.Another issue I hit was that under Vagrant with the shared filesystem,
pipwould notinstall into the virtual environment. It only will install when moving the PYTHON_ENV
off of the shared filesystem. So when provisining the Vagrant VM it will now put
PYTHON_ENV in C:\beats-python-env. Remember this if you need need to do a completely
clean build on Windows+Vagrant.
There were some encoding issues I found while reading output files containing unicode.
So I modified the libbeat's beat.py to specify
utf_8encoding when opening output files.I was seeing errors like: