Skip to content

[MRG] RealTime client refactor#6141

Merged
larsoner merged 18 commits intomne-tools:masterfrom
teonbrooks:refactor
Apr 23, 2019
Merged

[MRG] RealTime client refactor#6141
larsoner merged 18 commits intomne-tools:masterfrom
teonbrooks:refactor

Conversation

@teonbrooks
Copy link
Copy Markdown
Member

following up on #6120. It also adds a mock client and an example using LSL. This refactors the FieldTripClient and the lsl test.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 11, 2019

Codecov Report

Merging #6141 into master will not change coverage.
The diff coverage is 74.78%.

@@          Coverage Diff           @@
##           master   #6141   +/-   ##
======================================
  Coverage      89%     89%           
======================================
  Files         410     411    +1     
  Lines       73885   73885           
  Branches    12256   12249    -7     
======================================
  Hits        65765   65765           
- Misses       5224    5228    +4     
+ Partials     2896    2892    -4

@larsoner
Copy link
Copy Markdown
Member

This pull request introduces 10 alerts when merging 13c89af into 35d7c53 - view on LGTM.com

new alerts:

  • 5 for Unused import
  • 2 for Signature mismatch in overriding method
  • 2 for First argument of a method is not named 'self'
  • 1 for Unused local variable

Comment posted by LGTM.com

@larsoner
Copy link
Copy Markdown
Member

This pull request introduces 10 alerts when merging 8fba9e9 into aa74f02 - view on LGTM.com

new alerts:

  • 5 for Unused import
  • 2 for Signature mismatch in overriding method
  • 2 for First argument of a method is not named 'self'
  • 1 for Unused local variable

Comment posted by LGTM.com

@larsoner
Copy link
Copy Markdown
Member

This pull request introduces 1 alert when merging 6763f7f into 5610605 - view on LGTM.com

new alerts:

  • 1 for Wrong number of arguments in a call

Comment posted by LGTM.com

@teonbrooks teonbrooks mentioned this pull request Apr 16, 2019
@teonbrooks teonbrooks changed the title [WIP] RealTime client refactor [MRG] RealTime client refactor Apr 16, 2019
@larsoner
Copy link
Copy Markdown
Member

@jasmainak this is marked MRG now, can you take a look?

@teonbrooks teonbrooks force-pushed the refactor branch 2 times, most recently from 110dffe to a127971 Compare April 16, 2019 17:07
@jasmainak
Copy link
Copy Markdown
Member

I need to take a closer look by running some of the realtime examples. Could we mark this for the sprint?

@teonbrooks you should not tag someone in a commit if you plan to rebase :p I got like 10 emails because you rebased lol

@teonbrooks
Copy link
Copy Markdown
Member Author

teonbrooks commented Apr 16, 2019

@jasmainak i didn't tag you, lol. and i didn't realize you got emails about the rebasing too, sorry about that

"""

def __init__(self, host, n_channels=8, ch_type="eeg", sfreq=100,
testing=False):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead suggest that it take a raw object as an argument. Then you don't need n_channels, ch_type, sfreq etc.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, it should take any raw instance (subject to any necessary constraints of the streamer, e.g. only EEG channels or something) and "play it back"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is totally possible. looking at the code, i realize that the name mockrtclient is either a misnomer or it is working as both the server and the client. I feel that these two functionality should be separated. what i mean is that the client shouldn't be serving up data, it should be a listener that can pass the data along to other functionalities.

there should be stream generator that takes a raw file and send it to LSL or TCP/IP. The client should then be able to listen using their protocols. it could combine with the existing stim_server_client module for the latter protocol

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it would be cleaner to have a mock server emulate mne_rt_server but this is written in CPP. Writing a mock server to emulate this may not be so straightforward, and certainly was not for someone who had learned python 10 days ago ;-) But this could perhaps be a point of collaboration with @LorenzE since he may be familiar with the MNE-CPP code and the protocol for mne_rt_server.

@larsoner
Copy link
Copy Markdown
Member

This pull request introduces 2 alerts when merging 3f8f05e into b3dfd7a - view on LGTM.com

new alerts:

  • 2 for Unused import

Comment posted by LGTM.com

@jasmainak
Copy link
Copy Markdown
Member

@teonbrooks the CIs are not happy

I'll test tomorrow if the fieldtrip examples are not broken.

Otherwise, LGTM

@jasmainak
Copy link
Copy Markdown
Member

Also, you'd need to implement this to work with RtEpochs so that it is feature complete. We can open an issue when this PR is merged so that we don't forget

@jasmainak
Copy link
Copy Markdown
Member

Also, could we triage based on whether stream ID or stream name is provided? If this is not possible, perhaps we should add a bit of documentation how the stream ID can be obtained.

cc @akovrig

this is compatible across platforms
@jasmainak
Copy link
Copy Markdown
Member

LGTM. Whoever wants to feel productive this morning can merge :-)

@larsoner larsoner merged commit c9fa690 into mne-tools:master Apr 23, 2019
@larsoner
Copy link
Copy Markdown
Member

Thanks @teonbrooks @jasmainak !

@teonbrooks teonbrooks deleted the refactor branch April 23, 2019 09:23
@larsoner
Copy link
Copy Markdown
Member

Broke CircleCI:

generating gallery for auto_examples/realtime... [ 71%] plot_lslclient_rt.py
Note (minor): could not create multicast responder for address FF31:113D:6FDD:2C17:A643:FFE2:1BD1:3CD2 (failed with: set_option: No such device)
Note (minor): could not create multicast responder for address FF02:113D:6FDD:2C17:A643:FFE2:1BD1:3CD2 (failed with: set_option: No such device)
Note (minor): could not create multicast responder for address FF05:113D:6FDD:2C17:A643:FFE2:1BD1:3CD2 (failed with: set_option: No such device)
Process Process-1931:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/circleci/project/mne/realtime/mock_lsl_stream.py", line 81, in _initiate_stream
    outlet.push_sample(mysample)
  File "/home/circleci/.local/lib/python3.7/site-packages/pylsl/pylsl.py", line 454, in push_sample
    raise ValueError("length of the data must correspond to the "
ValueError: length of the data must correspond to the stream's channel count.
Stream transmission broke off (Input stream error.); re-connecting...

It gets killed shortly thereafter. Maybe there is a memory problem? CircleCI does not have a ton of memory. Can you fix it? Maybe it's not related to memory and is something else?

https://circleci.com/gh/mne-tools/mne-python/12424

massich pushed a commit to massich/mne-python that referenced this pull request Apr 23, 2019
* Add an example using the LSLClient

n_chan --> n_channels

* Refactor FTClient; Add MockLSLStream, refactor test to use mock stream

* update reference and whats new

* fixing some errors

* update style

* temp

* improvements to the realtime module

currently the test is breaking when it comes to using the RtEpochs object.

* minor fix

* move the RtEpochs testing to separate PR

* cleanup

* fix the way super is called

* updated the MockLSLStream to take raw instance

* add time dilation factor, cleanup

* add more info on lsl identifier

* address ci

* skip running test with multiprocessing on windows

Windows runs into a problem with multiprocessing: 'https://stackoverflow.com/questions/50079165/'

* cleaned up the windows check

* update the pylsl requirement to 1.12

this is compatible across platforms
@stfnrpplngr
Copy link
Copy Markdown
Contributor

Maybe a bit late, just stumbled upon this one:
https://github.com/kaczmarj/rteeg
@kaczmarj

@kaczmarj
Copy link
Copy Markdown
Contributor

@stfnrpplngr and others reading this - i just wanted to provide a disclaimer about https://github.com/kaczmarj/rteeg. i don't have time to maintain that package now, and frankly, i wrote that when i just started out programming, so it probably doesn't follow good practices...

i've just read through this PR, and i think it makes my rteeg package redundant. thanks @teonbrooks for all your work on this!

jeythekey pushed a commit to jeythekey/mne-python that referenced this pull request Apr 27, 2019
* Add an example using the LSLClient

n_chan --> n_channels

* Refactor FTClient; Add MockLSLStream, refactor test to use mock stream

* update reference and whats new

* fixing some errors

* update style

* temp

* improvements to the realtime module

currently the test is breaking when it comes to using the RtEpochs object.

* minor fix

* move the RtEpochs testing to separate PR

* cleanup

* fix the way super is called

* updated the MockLSLStream to take raw instance

* add time dilation factor, cleanup

* add more info on lsl identifier

* address ci

* skip running test with multiprocessing on windows

Windows runs into a problem with multiprocessing: 'https://stackoverflow.com/questions/50079165/'

* cleaned up the windows check

* update the pylsl requirement to 1.12

this is compatible across platforms
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants