feat: refactor and granulize traits #586
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ahirner
reviewed
Jul 19, 2025
ahirner
left a comment
There was a problem hiding this comment.
I was trying this branch and wonder if I should patch apalis-cron compile errors. Do you think core is kind of "stable"?
This was referenced Jul 21, 2025
Closed
ahirner
reviewed
Jul 22, 2025
ahirner
left a comment
There was a problem hiding this comment.
Thanks it compiles now!
I try to create a system where tasks are launched on a cron Schedule. Each task should run at most once. This could be done by a timeout condition or a concurrency limit of 1 per task.
The in-memory state should be queryable. It should inform much like systemd (last run outcome, current state, next schedule on .timer). Ulitmately, a health endpoint would inform about task failures, namely if any task failed.
I tried that with a relationship of one worker per cron stream and concurrency limit 1, all piping into a shared MemoryStorage. However, its new() form is not cloneable and its new_with_json() form obviously can't sink CronContext. Next, JsonMemory doesn't expose tasks to query from.
Maybe I'm holding the thing wrong. Is it simply better to implement a custom backend?
PS: looked a bit into Monitor as well, but it can't register tasks after starting the set of registered workers.
Comment on lines
399
to
401
| type Config = (); | ||
|
|
||
| type MakeError = String; |
There was a problem hiding this comment.
Somewhat related to the trait's Config. At least if MakeError could be infallible in this case, it would make constructing simple backends more ergonomic.
Member
Author
|
@ahirner Please note this is a work in progress.
|
geofmureithi
added a commit
that referenced
this pull request
Nov 18, 2025
* feat: refactor and granulize traits (#586) * fix: get working * fix: relax error trait to allow boxdyn * improve step builder * fix: make from_request async * fix: stabilze State * chore: refactor and free errors * fix: refactor ext * feat: add shared * fix: get shared working * feat: get everything working * complete core mods refactor * improve: api for workflow * Refactor changes * fix: improve dag flow * wip: improvements in the api * fix: more improvements in api * feat: improve stepped features * add: more improvements * fix: minor improvements * introduce more features * wip: continue on v1 changes * wip: more improvements * fix: make better call all * more: fixes on updates * fix: improve worker factory * move: workflow into its own crate * make serde features conditional * make serde features conditional * make futures features conditional * make tower features conditional * clippy: fixes * fixes: on api and tests * fix: more improvements * improve sinks * improve sinks * fix: request becomes task * fix: request becomes task * fix: improve TaskBuilder * introduce custom backends * improve and try generalize task_id * improve and try generalize task_id * feat: remove task_id constrain as ulid * improve: task_id fixes * wip: generic sinks and fetchers * improve metadata * docs: improvement * fix: get restarts working * feat: monitor can restart workers * refactor: remove backend with sink * checkpoint: trait step * checkpoint: better step trait * checkpoint: more workflow improbements * checkpoint: better backend * checkpoint: basic api * checkpoint: more improvements to steps * checkpoint: remove clones * checkpoint: filter_map * checkpoint: workflow improvements * chekpoint: improvements in impl * checkpoint: get tests working * checkpoint: more lint fixes * checkpoint: improve docs * improve: meta * fix: improve meta more * doc: improvements * checkpoint: retry policies * chore: make apalis crate compile * checkpoint: get layers working * improve tracing * checkpoint: generalize cron * checkpoint: docs no warnings * stabilized core * fix: improve core features * lint: cargo check * bump: version * chore: remove SECURITY.md * improvements on cron * chore: improve cron docs * chore: fix workflow * chore: add workflow docs * chore: add workflow docs * chore: fix workflow * chore: fix workflow * introduce: poll_strategy * improve features * fix: prelude * remove: crates no longer on this repo * chore: expose * chore: bump to 1.0.0-alpha.3 * minor: fixes on features * chore: improvements in the API * improve Backend API * api for expose * tiny api changes * tiny api changes * tiny api changes * fix: order of middleware * api: run_until and run_until_ctx functions for workers * fix: prevent retries if worker is shutting down * fix: prevent retries if worker is shutting down * improve status api * status meets retries * chore: queue api + some helpers * standardize: queue api * chore: minor features fix, cargo hack? * ease tasksink * split: list tasks * chore: add backend config basics * clippy: partial improvements * bump: to 1.0.0-alpha.4 * chore: get workflow working * include: link to board * add: acknowledgements * bump: to new versions * feat: introduce id_generator * improve: workflow api * improve service result for better typing * minor expose on step * minor pub on step * bump: version * reintroduce apalis-sql * bump: to new versions * remove excess keywords * remove excess category * improve: make monitor Send and Sync * chore: minor fixes + parallelization * bump: to new alpha version * bump: to new alpha version * improve: features_table macro to add relevant tests * improve Workflow test * improve docs generator * fix: features_table macro * fix: add codec type in serialization * chore: rename WorkFlow to Workflow * chore: bump to new version * chore: reintroduce dag in workflow * refactor: move crates to root * chore: cleanup CI * lint: cargo fmt * chore: CHANGELOG * refactor: crates, workflow and `BackendExt` (#623) * checkpoint: workflow fixes * refactor: workflow and backend ext * changelog: update * chore: bump to v1.0.0 beta.1 (#624) * chore: bump to v1.0-beta * changelog: update * lint: cargo fmt * chore: examples on v1 (#625) * chore: bump to v1.0-beta * changelog: update * lint: cargo fmt * chore: cleaup examples * lint: cargo clippy * fix: improve readme * introduce: contributing * chore: fix for cargo hack * chore: fix for cargo hack * fix: remove cyclic dependencies (#626) * chore: minor changelog (#629) * chore: remove self dep * chore: add pipe change * chore: include v0.7.4
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A big milestone is coming, a new version is in the kitchen. This comes with a refactor of the traits in the core.