Skip to content

[MRG] LSLClient#6120

Merged
larsoner merged 24 commits intomne-tools:masterfrom
teonbrooks:rtclient
Apr 11, 2019
Merged

[MRG] LSLClient#6120
larsoner merged 24 commits intomne-tools:masterfrom
teonbrooks:rtclient

Conversation

@teonbrooks
Copy link
Copy Markdown
Member

Got the urge to resurrect #4110 and maybe complete it. The aim of the PR is to create a realtime client for LSL that fits the way MNE handles Client objects. It also creates the basis for BaseClient class that will (in subsequent PRs) clean up the realtime module and make it easier to develop and use

@larsoner
Copy link
Copy Markdown
Member

larsoner commented Apr 4, 2019

This pull request introduces 2 alerts when merging f4d6d3a into 56909cd - view on LGTM.com

new alerts:

  • 1 for Except block handles 'BaseException'
  • 1 for Unused import

Comment posted by LGTM.com

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 4, 2019

Codecov Report

Merging #6120 into master will decrease coverage by 0.44%.
The diff coverage is 67.45%.

@@            Coverage Diff            @@
##           master   #6120      +/-   ##
=========================================
- Coverage   89.04%   88.6%   -0.45%     
=========================================
  Files         404     407       +3     
  Lines       73586   74160     +574     
  Branches    12218   12363     +145     
=========================================
+ Hits        65525   65709     +184     
- Misses       5188    5583     +395     
+ Partials     2873    2868       -5

@larsoner
Copy link
Copy Markdown
Member

larsoner commented Apr 4, 2019

This pull request introduces 1 alert when merging 562c932 into 56909cd - view on LGTM.com

new alerts:

  • 1 for Unused local variable

Comment posted by LGTM.com

@teonbrooks teonbrooks changed the title [WIP] LSLClient [MRG] LSLClient Apr 4, 2019
@larsoner
Copy link
Copy Markdown
Member

larsoner commented Apr 4, 2019

This pull request introduces 1 alert when merging 0263807 into 56909cd - view on LGTM.com

new alerts:

  • 1 for Unused import

Comment posted by LGTM.com

@jona-sassenhagen
Copy link
Copy Markdown
Contributor

lol @ commit 0263807

@jona-sassenhagen
Copy link
Copy Markdown
Contributor

Very cool to have this functionality!

@larsoner
Copy link
Copy Markdown
Member

larsoner commented Apr 5, 2019

This pull request introduces 1 alert when merging 78a72f4 into 027f82d - view on LGTM.com

new alerts:

  • 1 for Unused import

Comment posted by LGTM.com

@jasmainak
Copy link
Copy Markdown
Member

@teonbrooks do you think we could make FieldTripClient use _BaseClient ? It would help iron out potential issues / shortcomings and inconsistencies

@jasmainak
Copy link
Copy Markdown
Member

also don't forget to update whats_new.rst

@jasmainak
Copy link
Copy Markdown
Member

@teonbrooks sounds exciting! I hope we can get it merged soon

@jasmainak
Copy link
Copy Markdown
Member

@teonbrooks the CI is not happy!

@teonbrooks
Copy link
Copy Markdown
Member Author

anyone with any idea why one of the travis builds is just becoming unresponsive? it's the one with py3.6 as req. am i doing something wrong with specifying the test build environment?

@teonbrooks teonbrooks force-pushed the rtclient branch 2 times, most recently from f2e7e39 to e61fe4a Compare April 7, 2019 02:43
@larsoner
Copy link
Copy Markdown
Member

larsoner commented Apr 7, 2019

The file mne/realtime/tests/_start_fake_lsl_stream.py has a while loop, it looks suspiciously like something that could make things get stuck during test collection if pytest tries to import mne/realtime/tests/_start_fake_lsl_stream.py. I'd put all that stuff in a function.

@teonbrooks
Copy link
Copy Markdown
Member Author

so what I noticed it rewriting this tests several times is that the errorring is happening because of the incoming network notification Mac produce when I am launching the mock lsl stream. when running the scripts, the kernel doesn't get permission and then the client connection fails. I'm able to run the server when it is in interactive python, but not as a script

change the pattern of checking for the import following common pattern seen in mne
forgot that I changed the class signature. If this doesn't work, I will implement multiprocessing to create the LSL Stream for the test
just caught the fact that I didn't change the base class to take the private connect function
Fix a typo in the LSLClient._connect which led to downstream errors.
Copy link
Copy Markdown
Member

@jasmainak jasmainak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After unrelenting pressure from @teonbrooks I had to approve this PR. But he has committed to improve things in the following PR #6141. This PR is already improving the state of things, so I don't want to block it any further.

@agramfort or @larsoner , if you are happy feel free to merge.

Copy link
Copy Markdown
Member

@larsoner larsoner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed a commit to add PyLSL to the requirements lists in two other places. Otherwise LGTM. Will merge once CIs come back happy.

Locally on Linux x86_64 Python 3.7 pip refused to install PyLSL>=1.13.1 saying:

Could not find a version that satisfies the requirement pylsl>=1.13.1 (from versions: 1.10.3, 1.10.4, 1.10.5, 1.12.2)

So I wouldn't be surprised if the 3.7-pip Linux x86_64 build also fails. This could be a packaging problem for PyLSL or a problem locally with my pip (or commands).

@larsoner
Copy link
Copy Markdown
Member

Indeed CircleCI, which is 3.7, confirms they have a packaging problem;

https://circleci.com/gh/mne-tools/mne-python/12197?utm_campaign=workflow-failed&utm_medium=email&utm_source=notification

@teonbrooks do you work with the PyLSL folks at all? Can you look into it? It is listed as supposedly compatible here:

https://pypi.org/project/pylsl/

@larsoner
Copy link
Copy Markdown
Member

Actually looks like it's a known issue:

labstreaminglayer/pylsl#6

I'll comment it out in requirements.txt for now

@larsoner larsoner merged commit 5610605 into mne-tools:master Apr 11, 2019
@larsoner
Copy link
Copy Markdown
Member

Thanks @teonbrooks

@teonbrooks teonbrooks deleted the rtclient branch April 11, 2019 20:20
@larsoner larsoner mentioned this pull request Apr 13, 2019
@cbrnr
Copy link
Copy Markdown
Contributor

cbrnr commented Apr 16, 2019

Is there an example where people can see how this LSL client works (ideally including online visualization of the incoming data)?

@teonbrooks
Copy link
Copy Markdown
Member Author

@cbrnr, yes, it's part of #6141. @jasmainak didn't like the explicit call to generate a mock lsl stream ;). I didn't want a patched solution, so I made a proper mock stream class. I ran into a problem working with the RtEpochs class. I might need to split that investigation from the pending PR and tackle that next week at the sprint

@larsoner
Copy link
Copy Markdown
Member

It's possible that the realtime code could use some fixes / cleanup, this would be great for the sprint

@teonbrooks
Copy link
Copy Markdown
Member Author

It's possible that the realtime code could use some fixes / cleanup, this would be great for the sprint

my thoughts exactly. I opened an issue (#6157) and tacked it on to the Project board.

@jasmainak
Copy link
Copy Markdown
Member

I think it would be cool to emulate one of the FieldTrip examples with pylsl. So we can show the evoked update as the data comes in.

@jasmainak
Copy link
Copy Markdown
Member

Also, if we could have the realtime examples run in CircleCI that would be great. So that I don't have to run it on my computer every time.

@larsoner
Copy link
Copy Markdown
Member

In theory we should be able to unify the structure for running a fake server from Travis / unit-tests for use by plot_realtime_* functions. But please make sure these run as quickly as possible (definitely < 1 min, preferably in seconds). We might want for example a FakeServer(..., time_dilation=1) parameter that we can set to time_dilation=10 in the example so that 30 sec of data get fed to the client in 3 sec.

@jasmainak
Copy link
Copy Markdown
Member

yes agreed. In the fieldtrip buffer, you can have a speedup factor so that it runs very fast.

jeythekey pushed a commit to jeythekey/mne-python that referenced this pull request Apr 27, 2019
* working on the realtime client

* new base class for rtclient

* updates to the base client

* add import

* first pass at lsl in mne

* UPDATE lsl client, base_client, FIX epochs

* decouple the refactor from the LSLClient implementation

this reverts the attempt to both add an LSLClient and create a BaseClient class. For this upcoming PR, the focus will be adding support to LSL. subsequent PRs can be done to refactor the realtime module

* update to lsl client

at the moment, I am able to connect to a stream.
I need to test whether I am able to read the data from the stream.
opened an issue on pylsl to inquire whether i can both send and receive on same machine for local dev purpose (labstreaminglayer/pylsl#10)

once I can confirm the data is coming in properly, i will add some tests

* Update lsl_client.py

mistakenly tried to create inlets twice. this is the mvp for the LSL

* code cleanup and the beginning of tests for the lslclient

* Update test_lsl_client.py

* starting to remember the pythonic way of coding

been living in the world of javascript. feeling like a wayward youth

* simplify time

* Update lsl_client.py

* address some comment in the pr

change the pattern of checking for the import following common pattern seen in mne

* [FIX] _testing.py: copy pasted but never corrected

* Update test_lsl_client.py

forgot that I changed the class signature. If this doesn't work, I will implement multiprocessing to create the LSL Stream for the test

* testing is very useful

just caught the fact that I didn't change the base class to take the private connect function

* FIX arg in _connect; Implement thread to test

Fix a typo in the LSLClient._connect which led to downstream errors.

* address styling

* replace thread with process for clean exit

* linting complete

* FIX: Add to two other req lists

* FIX: Not on PyPi for Linux
@cboulay
Copy link
Copy Markdown

cboulay commented Jul 23, 2020

FYI, I finally got around to updating the CI scripts so Appveyor automatically builds and distributes pylsl for all major platforms. There's a new version now on pypi. Please let me know if you encounter any problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants