Skip to content

[python] Add independent deployment for python gateway server#7549

Merged
caishunfeng merged 8 commits intoapache:devfrom
zhongjiajie:py-deploy-alone
Jan 7, 2022
Merged

[python] Add independent deployment for python gateway server#7549
caishunfeng merged 8 commits intoapache:devfrom
zhongjiajie:py-deploy-alone

Conversation

@zhongjiajie
Copy link
Copy Markdown
Member

@zhongjiajie zhongjiajie commented Dec 22, 2021

For now, pythonGatewayServer could only run in a standalone server, and standalone is not for the production environment, we should make pythonGatewayServer work like api-server or master-server. We should also make python gateway server work in docker/K8S enviroment.

close: #7539

@zhongjiajie zhongjiajie changed the title [python] Add independent deployment to python gateway server [WIP][python] Add independent deployment to python gateway server Dec 22, 2021
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Dec 22, 2021

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 41.02%. Comparing base (7fd94ff) to head (87b4d79).
⚠️ Report is 3135 commits behind head on dev.

Additional details and impacted files
@@             Coverage Diff              @@
##                dev    #7549      +/-   ##
============================================
- Coverage     41.03%   41.02%   -0.02%     
+ Complexity     3695     3693       -2     
============================================
  Files           637      637              
  Lines         27025    27025              
  Branches       3063     3063              
============================================
- Hits          11089    11086       -3     
- Misses        14867    14872       +5     
+ Partials       1069     1067       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@zhongjiajie zhongjiajie force-pushed the py-deploy-alone branch 3 times, most recently from 43fb0c5 to c82e27e Compare December 24, 2021 06:33
@zhongjiajie
Copy link
Copy Markdown
Member Author

We could success build and the result as below
image

@zhongjiajie zhongjiajie marked this pull request as ready for review December 24, 2021 06:37
@zhongjiajie zhongjiajie added the feature new feature label Dec 29, 2021
@zhongjiajie zhongjiajie requested review from CalvinKirs and removed request for CalvinKirs January 5, 2022 11:31
@zhongjiajie zhongjiajie self-assigned this Jan 5, 2022
@zhongjiajie zhongjiajie changed the title [WIP][python] Add independent deployment to python gateway server [python] Add independent deployment to python gateway server Jan 5, 2022
@zhongjiajie zhongjiajie changed the title [python] Add independent deployment to python gateway server [python] Add independent deployment for python gateway server Jan 6, 2022
@zhongjiajie zhongjiajie requested a review from lenboo January 6, 2022 01:46
caishunfeng
caishunfeng previously approved these changes Jan 6, 2022
Copy link
Copy Markdown
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhongjiajie
Copy link
Copy Markdown
Member Author

zhongjiajie commented Jan 6, 2022

I already test

  • Using dolphinscheduler-daemon.sh start python-gateway-server to start server
  • Using ./bin/install to
    • Install in single machine
    • Install in multiple machine

Configure to other database failed, but is not related to this PR, So I think it is ready to go now

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Jan 6, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

Copy link
Copy Markdown
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@caishunfeng caishunfeng merged commit da4a859 into apache:dev Jan 7, 2022
@zhongjiajie zhongjiajie deleted the py-deploy-alone branch January 7, 2022 07:16
@zhongjiajie
Copy link
Copy Markdown
Member Author

Thanks

zhongjiajie added a commit to zhongjiajie/dolphinscheduler that referenced this pull request Apr 6, 2022
…apache#7549)

* [python] Add independent deployment to python gateway server

* Correct some error

* Add missing deployment change

* Add missing expose port

* Add missing standalone config

* Keyword change from rpc to socket

* try to fix

* Add python gateway server to scp-hosts.sh (da4a859)
zhongjiajie added a commit to zhongjiajie/dolphinscheduler that referenced this pull request Apr 6, 2022
Currently the size of our distribute package is up to
800MB, this patch is migrate python gateway server into
api server

Revert python-gateway server related change

Revert: [python] Add independent deployment for python gateway server (apache#7549)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][python] Gateway deployment in production mode

3 participants