Skip to content

Add websockets for streaming to a frontend#1414

Closed
Tylersuard wants to merge 0 commit into
microsoft:mainfrom
Tylersuard:main
Closed

Add websockets for streaming to a frontend#1414
Tylersuard wants to merge 0 commit into
microsoft:mainfrom
Tylersuard:main

Conversation

@Tylersuard

@Tylersuard Tylersuard commented Jan 26, 2024

Copy link
Copy Markdown

@sonichi

Why are these changes needed?

If a user wants to have a frontend for an autogen-based chat, websockets will be required to stream information out of autogen. These changes will allow the user to stream content from a conversation to any address they choose. See https://websockets.readthedocs.io/en/stable/

Related issue number

#1199 , #1143 , #217 , #394

Checks

@Tylersuard

Tylersuard commented Jan 26, 2024

Copy link
Copy Markdown
Author

@sonichi the code formatting test is failing on scripts that I didn't write. Also, I have no idea why adding websockets would interfere with the WebSurfer tests, but it's happening with other PR's as well.

@codecov-commenter

codecov-commenter commented Jan 26, 2024

Copy link
Copy Markdown

Codecov Report

Attention: 14 lines in your changes are missing coverage. Please review.

Comparison is base (015070b) 33.12% compared to head (76c8770) 4.17%.

Files Patch % Lines
autogen/agentchat/conversable_agent.py 0.00% 6 Missing ⚠️
autogen/oai/client.py 33.33% 4 Missing ⚠️
autogen/streaming_utils.py 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #1414       +/-   ##
==========================================
- Coverage   33.12%   4.17%   -28.95%     
==========================================
  Files          42      41        -1     
  Lines        5051    5054        +3     
  Branches     1157    1225       +68     
==========================================
- Hits         1673     211     -1462     
- Misses       3250    4833     +1583     
+ Partials      128      10      -118     
Flag Coverage Δ
unittests 4.17% <12.50%> (-28.91%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Tylersuard

Copy link
Copy Markdown
Author

@ekzhu Is this more along the lines of what you were thinking?

Comment thread autogen/agentchat/conversable_agent.py Outdated
Comment thread autogen/agentchat/conversable_agent.py Outdated
self._oai_messages = defaultdict(list)
self._oai_system_message = [{"content": system_message, "role": "system"}]
self.description = description if description is not None else system_message
self.streaming_transmitter = streaming_transmitter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.streaming_transmitter = streaming_transmitter
self._streaming_transmitter = streaming_transmitter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use private as it is not really called directly from outside of the class.

Comment thread autogen/extensions/streaming_transmitter.py Outdated
Comment thread autogen/extensions/websocket_streaming_transmitter.py Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is not needed any more.

@ekzhu ekzhu Jan 31, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create tests in test/agentchat/test_conversable_agent_streaming.py.

Comment thread autogen/oai/client.py Outdated
self,
*,
config_list: Optional[List[Dict[str, Any]]] = None,
streaming_transmitter: Optional[StreamingTransmitter] = None,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

streaming_transmitter goes into the extra_kwargs list see above. It will be taken out of the base_config dictionary in the constructor. See how cache is used in create

@ekzhu

ekzhu commented Jan 31, 2024

Copy link
Copy Markdown
Contributor

@ekzhu Is this more along the lines of what you were thinking?

Thanks for the update! Yes this is what I was thinking. We don't need to have separate WebSocketStreamingTransmitter as the websocket class already implements the StreamingTransmitter protocol.

@ragyabraham @joshkyh @victordibia @davorrunje does this current setup for streaming make sense to your guys from application point of view?

Comment thread autogen/oai/client.py

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A giant PR is coming its way changing this file completely. #1345.

So perhaps it is a good idea to wait until that one gets merged before continue working on this one.

@joshkyh

joshkyh commented Jan 31, 2024

Copy link
Copy Markdown
Contributor

does this current setup for streaming make sense to your guys from application point of view?

I'll defer to @ragyabraham, as I don't work usually work with sockets. :)

Comment thread autogen/extensions/streaming_transmitter.py Outdated
@davorrunje

davorrunje commented Feb 1, 2024

Copy link
Copy Markdown
Contributor

@Tylersuard do you want me to look into failing tests?

@ekzhu

ekzhu commented Feb 4, 2024

Copy link
Copy Markdown
Contributor

looks like we might be getting a dangling PR here.

@davorrunje would you be interested in taking over this?

@Tylersuard

Tylersuard commented Feb 4, 2024 via email

Copy link
Copy Markdown
Author

@ekzhu

ekzhu commented Feb 4, 2024

Copy link
Copy Markdown
Contributor

@Tylersuard you can see my comments in the code. There is a recent change to the oai/client.py that conflicts with the changes you made. So there is work in redoing your changes but in the new oai/client.py. There are a couple minor comments I have that could be addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants