Skip to content

feat: [#651] add machinery driver#1055

Merged
hwbrzzl merged 4 commits intomasterfrom
bowen/#651-2
May 31, 2025
Merged

feat: [#651] add machinery driver#1055
hwbrzzl merged 4 commits intomasterfrom
bowen/#651-2

Conversation

@hwbrzzl
Copy link
Contributor

@hwbrzzl hwbrzzl commented May 31, 2025

📑 Description

Closes goravel/goravel#651

Previoushly, we ddin't want to push task to the machinery driver any more, only pop old tasks from the machinery driver and push new tasks to the new redis driver. But if so, it's a break-change for user, if there are any issue in the new redis driver, it's not good for users.

So in this PR, I added the machinery driver back, let user can continous use the machinery driver, they can swith the driver to redis in v1.17.

This pull request introduces a new machinery driver for the queue package, along with significant updates to the Machinery implementation and its integration. It also includes updates to the DriverCreator and PendingJob classes to support the new driver and improve logging capabilities. The changes are accompanied by comprehensive tests for the new functionality.

New Driver Implementation and Integration:

  • Added a new machinery driver constant to the queue/driver.go file. ([contracts/queue/driver.goR6](https://github.com/goravel/framework/pull/1055/files#diff-74bbb8b0d13cc6315ecbf19d7e06d4e345c020d2065533d5341590c0a81c493aR6))
  • Implemented the Machinery driver in queue/driver_machinery.go, including methods for pushing tasks, running workers, and handling task chains. Simplified the initialization by dynamically creating the server configuration and broker/queue setup. ([[1]](https://github.com/goravel/framework/pull/1055/files#diff-5ffcad5fdc36c9ed9f2960478a6673b66e07faaf1e586446fab5b9bdce62c58aL8-R28), [[2]](https://github.com/goravel/framework/pull/1055/files#diff-5ffcad5fdc36c9ed9f2960478a6673b66e07faaf1e586446fab5b9bdce62c58aL44-L60), [[3]](https://github.com/goravel/framework/pull/1055/files#diff-5ffcad5fdc36c9ed9f2960478a6673b66e07faaf1e586446fab5b9bdce62c58aL69-R107), [[4]](https://github.com/goravel/framework/pull/1055/files#diff-5ffcad5fdc36c9ed9f2960478a6673b66e07faaf1e586446fab5b9bdce62c58aL121-R165))

Updates to DriverCreator:

  • Updated DriverCreator to include a log dependency and support the new machinery driver. ([[1]](https://github.com/goravel/framework/pull/1055/files#diff-96facd005dd2e2be2900606a7b0b4395a8c9769c7f4f3e46c8ff8d60c39d6adeR6), [[2]](https://github.com/goravel/framework/pull/1055/files#diff-96facd005dd2e2be2900606a7b0b4395a8c9769c7f4f3e46c8ff8d60c39d6adeR16-R25), [[3]](https://github.com/goravel/framework/pull/1055/files#diff-96facd005dd2e2be2900606a7b0b4395a8c9769c7f4f3e46c8ff8d60c39d6adeR41-R42))
  • Adjusted DriverCreatorTestSuite to include tests for the machinery driver. ([[1]](https://github.com/goravel/framework/pull/1055/files#diff-36597f12e55c9723a3d81464173012d57487b6411d43c2d85cb085df070fbbb7L42-R42), [[2]](https://github.com/goravel/framework/pull/1055/files#diff-36597f12e55c9723a3d81464173012d57487b6411d43c2d85cb085df070fbbb7R90-R105))

Logging Enhancements:

  • Added log as a dependency to PendingJob and its associated NewPendingJob constructor to improve logging capabilities. ([[1]](https://github.com/goravel/framework/pull/1055/files#diff-6cfb98ddf9021590d4029ab469f40e58403d7a929bf8874001736ddc2e10d319R9), [[2]](https://github.com/goravel/framework/pull/1055/files#diff-6cfb98ddf9021590d4029ab469f40e58403d7a929bf8874001736ddc2e10d319R28), [[3]](https://github.com/goravel/framework/pull/1055/files#diff-6cfb98ddf9021590d4029ab469f40e58403d7a929bf8874001736ddc2e10d319L38-R40))
  • Integrated logging into the Machinery driver for debugging, informational messages, and error handling. ([[1]](https://github.com/goravel/framework/pull/1055/files#diff-5ffcad5fdc36c9ed9f2960478a6673b66e07faaf1e586446fab5b9bdce62c58aL8-R28), [[2]](https://github.com/goravel/framework/pull/1055/files#diff-5ffcad5fdc36c9ed9f2960478a6673b66e07faaf1e586446fab5b9bdce62c58aL69-R107))

Configuration Improvements:

  • Enhanced queue configuration in queue/application.go to dynamically fetch queue and concurrency settings from the configuration file. ([queue/application.goL68-R73](https://github.com/goravel/framework/pull/1055/files#diff-34aafc02eb2c7a5ca85c1a4a79bb1f8e58711850cc764d4fc58e96401b397bb3L68-R73))

Test Coverage:

  • Added comprehensive unit tests for the Machinery driver in queue/driver_machinery_test.go, including tests for task pushing, running workers, handling delays, and task chains. ([[1]](https://github.com/goravel/framework/pull/1055/files#diff-427d61adc322d75e940292154357e58dfeb4c9bfcabcb445b9ef43584b786b4eR4-R24), [[2]](https://github.com/goravel/framework/pull/1055/files#diff-427d61adc322d75e940292154357e58dfeb4c9bfcabcb445b9ef43584b786b4eL38-R239))

✅ Checks

  • Added test cases for my code

@codecov
Copy link

codecov bot commented May 31, 2025

Codecov Report

Attention: Patch coverage is 87.64045% with 11 lines in your changes missing coverage. Please review.

Project coverage is 71.42%. Comparing base (f6542cd) to head (b3aea5b).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
queue/driver_machinery.go 93.15% 4 Missing and 1 partial ⚠️
queue/worker.go 20.00% 4 Missing ⚠️
queue/application.go 50.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1055      +/-   ##
==========================================
+ Coverage   71.21%   71.42%   +0.20%     
==========================================
  Files         178      178              
  Lines       12548    12587      +39     
==========================================
+ Hits         8936     8990      +54     
+ Misses       3220     3209      -11     
+ Partials      392      388       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hwbrzzl hwbrzzl marked this pull request as ready for review May 31, 2025 06:50
Copilot AI review requested due to automatic review settings May 31, 2025 06:50
@hwbrzzl hwbrzzl requested a review from a team as a code owner May 31, 2025 06:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR reintroduces the machinery driver for the queue package to allow continued usage of the previous driver alongside the new Redis driver, thereby mitigating break-changes for users upgrading to v1.17. Key changes include

  • Adding and integrating the new machinery driver (including logging enhancements)
  • Updating DriverCreator and PendingJob implementations to support the additional log dependency
  • Enhancing configuration and test coverage for both driver and chaining functionality

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
queue/worker.go Updated driver creator usage and conditional branching for machinery
queue/pending_job*.go Adjusted constructor parameters to accept log dependency
queue/driver_machinery*.go Refactored driver implementation and added chain support
queue/driver_creator*.go Extended driver creator to support machinery driver creation
queue/application.go Updated job submission methods to include log dependency
contracts/queue/driver.go Added new DriverMachinery constant

@hwbrzzl hwbrzzl merged commit 1d20e34 into master May 31, 2025
14 checks passed
@hwbrzzl hwbrzzl deleted the bowen/#651-2 branch May 31, 2025 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add DB driver for Queue module

2 participants