Skip to content

persist params to a file if requested#72

Merged
evgeni merged 1 commit intomasterfrom
persist
May 27, 2025
Merged

persist params to a file if requested#72
evgeni merged 1 commit intomasterfrom
persist

Conversation

@evgeni
Copy link
Member

@evgeni evgeni commented Apr 30, 2025

Summary by CodeRabbit

  • New Features

    • Added support for persisting command-line parameters across runs, with options to reset or exclude specific parameters.
    • Enhanced argument parsing to allow loading and resetting of persisted parameters via new reset-style flags.
    • Introduced new playbook variables for database configuration, including conditional validation and automatic reset behavior.
  • Bug Fixes

    • Corrected environment variable checks for enabling extra variables.
  • Documentation

    • Updated playbook metadata schema to document new database configuration options and reset rules.

@evgeni
Copy link
Member Author

evgeni commented Apr 30, 2025

This is a naive approach to persisting parameters, which has a few flaws that I want to point out before continuing:

  • Python's argparse only performs type validation/conversion if the passed in thing is a string (as that's what you get from a shell), so if we store some things as native types (like a obsah.data_types.Port as a Integer), it won't be validated again (see https://docs.python.org/3/library/argparse.html#type) -- to force re-validation we need to store things as strings (shouldn't be hard to implement, just didn't do it yet)
  • It assumes all actions take the same (or a subset of) params -- I think that'll be true for our installer effort, but is not for obal. This should be fine (we don't need persistence in obal), but worth pointing out
  • There is no way to unset a parameter. This is especially annoying when we have mutually exclusive parameters (hostname and url in the types playbook), as it means once you pass in one, you can never pass in the other. I think that's not acceptable, but I also hate the whole --reset-* wall-of-params we have with Kafo.

@ehelms
Copy link
Member

ehelms commented Apr 30, 2025

  • It assumes all actions take the same (or a subset of) params -- I think that'll be true for our installer effort, but is not for obal. This should be fine (we don't need persistence in obal), but worth pointing out

What if there are params unique to a specific command? This would not work? I'm not sure I have a good example yet, just looking for clarity.

  • There is no way to unset a parameter. This is especially annoying when we have mutually exclusive parameters (hostname and url in the types playbook), as it means once you pass in one, you can never pass in the other. I think that's not acceptable, but I also hate the whole --reset-* wall-of-params we have with Kafo.

This one I am not sure I follow as much. What is preventing unsetting of a parameter?

@evgeni
Copy link
Member Author

evgeni commented Apr 30, 2025

  • It assumes all actions take the same (or a subset of) params -- I think that'll be true for our installer effort, but is not for obal. This should be fine (we don't need persistence in obal), but worth pointing out

What if there are params unique to a specific command? This would not work? I'm not sure I have a good example yet, just looking for clarity.

If the param is specific to that one command, and not bothered by the others, it'd be fine (it will be saved, restored, and carried on, the other commands will just ignore it).
It gets weird if the same param means different things to different commands.

  • There is no way to unset a parameter. This is especially annoying when we have mutually exclusive parameters (hostname and url in the types playbook), as it means once you pass in one, you can never pass in the other. I think that's not acceptable, but I also hate the whole --reset-* wall-of-params we have with Kafo.

This one I am not sure I follow as much. What is preventing unsetting of a parameter?

The lack of knowledge when to unset it. There is no --reset-<parameter> flag to revert to the default and we have nothing in place that goes "oh, they passed --url now, so I will unset --hostname as that's exactly what the user meant"

@ehelms
Copy link
Member

ehelms commented Apr 30, 2025

It gets weird if the same param means different things to different commands.

Hrmm, I wonder about some logic to detect the same parameter defined twice (not inherited).

The lack of knowledge when to unset it. There is no --reset-<parameter> flag to revert to the default and we have nothing in place that goes "oh, they passed --url now, so I will unset --hostname as that's exactly what the user meant"

Oh I see. If they had passed in --database-host database.example.com --port 3456 and now they want to revert to the default by passing --database-host it would keep the --port 3456 value. I'd lean towards make them pass in required parameters together always, even if a value isn't changing. That's what it currently does without saving of params.

@evgeni
Copy link
Member Author

evgeni commented Apr 30, 2025

I'd lean towards make them pass in required parameters together always, even if a value isn't changing. That's what it currently does without saving of params.

That means we don't need to save params at all? :)

I have a feeling we don't talk about the same thing tho.

Given theforeman/foremanctl#131, what params should be persisted when someone does ./dop deploy <…>

@ehelms
Copy link
Member

ehelms commented Apr 30, 2025

@evgeni
Copy link
Member Author

evgeni commented May 2, 2025

Okay, then follow up questions!

Assume the user deployed with --database-mode external --database-ssl-mode verify-full --database-ssl-ca /etc/my/CArona --database-host psql.example.com, what happens…

  • with database_ssl_ca when they switch database_ssl_mode to disable?
  • with database_host when they switch database_mode to internal?

While the former is probably benign, the later is interesting: you set mode=internal, which would probably trigger the deployment of a local PostgreSQL container and the creation of the DBs/users, but the application still gets passed host=psql.example.com from database_host as that never got unset. The same would also happen if the user does never set mode=external, but does pass in database_host.

To prevent the user-input-error, you could enhance the constraints from

constraints:
  required_together:
    - [database_ssl_mode, database_ssl_ca]
  required_if:
    - ['database_mode', 'external', ['database_host', 'database_port']]

to

constraints:
  required_together:
    - [database_ssl_mode, database_ssl_ca]
  required_if:
    - ['database_mode', 'external', ['database_host', 'database_port']]
  forbidden_if:
    - ['database_mode', 'internal', ['database_host', 'database_port']]

But that would yield the issue I was talking about above when the user switches from a correct mode=external deployment, to internal, as database_host remains defined (because we persisted it), yet the mode=internal makes it forbidden.

@ehelms
Copy link
Member

ehelms commented May 2, 2025

  • with database_ssl_ca when they switch database_ssl_mode to disable?

We can choose to unset it or leave the value since it won't be used. I lean towards the former.

  • with database_host when they switch database_mode to internal?

I would choose to unset the database-host and make it revert to it's default value.

This would mean we need a mechanism to define what get's set and unset based on changes to values. The alternative is we don't get crazy with this logic and we just document things.

@evgeni
Copy link
Member Author

evgeni commented May 7, 2025

Unsetting would require more logic which we don't have today.

We could do the same we do for certs: have different vars files for different deployment styles.
pro: we could only map the needed things for internal db (aka: always leaving host at localhost)
contra: if we introduce new stuff, we need to update both mappings

@evgeni evgeni force-pushed the persist branch 4 times, most recently from a75c92b to 9d9f072 Compare May 12, 2025 10:28
@evgeni evgeni force-pushed the persist branch 2 times, most recently from fa82565 to 4a36eb9 Compare May 13, 2025 10:59
@evgeni evgeni marked this pull request as ready for review May 19, 2025 11:56
@evgeni
Copy link
Member Author

evgeni commented May 19, 2025

I think this is now ready for review

Copy link
Member

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must admit I only looked at the path storage now and not the other bits.

"""
Where to persist parameters to
"""
return os.environ.get('OBSAH_PERSIST_PATH', '/var/lib/obsah/parameters.yaml')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd lean to introducing a state directory and make this state_dir / 'parameters.yaml'. This may be a bit of bike shedding, but perhaps respect XDG_STATE_HOME so it can work out of the box in test setups as well. Then in rop packaging we should override it with OBSAH_STATE_DIR=/var/lib/rop

Copy link
Member

@ehelms ehelms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some lightweight testing and this looked good.

@evgeni
Copy link
Member Author

evgeni commented May 20, 2025

@coderabbitai review

@coderabbitai
Copy link

coderabbitai bot commented May 20, 2025

"""

Walkthrough

The changes introduce a mechanism for persisting command-line parameters across runs, allowing selective resetting and conditional persistence. New static methods and functions manage persistence behaviors and file locations. The argument parser and main control flow are updated to load, reset, and save parameters as specified. Playbook metadata and test fixtures are extended to support new variables, constraints, and reset rules.

Changes

File(s) Change Summary
obsah/init.py Added static methods state_path(), persist_params(), and persist_path() to ApplicationConfig for parameter persistence control. Enhanced argument parser to load persisted parameters, add reset flags, and track non-persisted arguments. Added reset_args() function to handle resetting persisted parameters based on metadata and flags. Modified main() to integrate persistence logic and update parameter saving. Updated Playbook.metadata to include a reset key.
tests/fixtures/playbooks/types/...yaml Added database_mode and database_host variables with help descriptions and choices. Introduced conditional constraints: required_if for requiring database_host when database_mode is external, and forbidden_if for forbidding database_host when database_mode is internal. Added a reset rule to clear database_host when database_mode changes.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant ApplicationConfig
    participant PersistenceFile
    participant Playbook

    User->>CLI: Run obsah command with arguments
    CLI->>ApplicationConfig: Check if persistence is enabled
    ApplicationConfig-->>CLI: Return persistence status and file path
    CLI->>PersistenceFile: Load persisted parameters (if enabled)
    CLI->>CLI: Merge persisted parameters as defaults
    CLI->>CLI: Parse command-line arguments
    CLI->>Playbook: Load metadata (including reset rules)
    CLI->>CLI: Apply reset_args() to handle reset logic
    CLI->>CLI: Validate arguments and inventory
    CLI->>PersistenceFile: Save updated parameters (excluding non-persisted)
    CLI-->>User: Execute playbook with final parameters
Loading

Poem

In a warren of code, persistence now grows,
Arguments remembered, as the bunny well knows.
With a hop and a skip, reset rules take flight,
Database modes shifting, hosts set just right.
🐇 YAML and Python, in harmony they dwell—
A clever new feature, the rabbit will tell!
"""

Note

⚡️ AI Code Reviews for VS Code, Cursor, Windsurf

CodeRabbit now has a plugin for VS Code, Cursor and Windsurf. This brings AI code reviews directly in the code editor. Each commit is reviewed immediately, finding bugs before the PR is raised. Seamless context handoff to your AI code agent ensures that you can easily incorporate review feedback.
Learn more here.


Note

⚡️ Faster reviews with caching

CodeRabbit now supports caching for code and dependencies, helping speed up reviews. This means quicker feedback, reduced wait times, and a smoother review experience overall. Cached data is encrypted and stored securely. This feature will be automatically enabled for all accounts on May 30th. To opt out, configure Review - Disable Cache at either the organization or repository level. If you prefer to disable all data retention across your organization, simply turn off the Data Retention setting under your Organization Settings.
Enjoy the performance boost—your workflow just got faster.

✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
obsah/__init__.py (2)

260-265: Persist-path should respect per-user state locations

Hard-coding /var/lib/obsah/parameters.yaml limits usability for non-root users (e.g. CI pipelines, local development).
Consider falling back to XDG_STATE_HOME (or $HOME/.local/state) when OBSAH_PERSIST_PATH is unset.

-        return os.environ.get('OBSAH_PERSIST_PATH', '/var/lib/obsah/parameters.yaml')
+        default_dir = (
+            os.environ.get('XDG_STATE_HOME')
+            or os.path.join(os.path.expanduser('~'), '.local', 'state')
+        )
+        return os.environ.get(
+            'OBSAH_PERSIST_PATH',
+            os.path.join(default_dir, 'obsah', 'parameters.yaml'),
+        )

366-373: Positional parameters are silently excluded from persistence

When variable.parameter is positional (no leading --), the code adds the
variable name to obsah_dont_persist, effectively making positional
arguments never persistable. This is intentional for target, but it may
surprise users for other positionals.

If the design goal is never to persist positionals, add a short comment to
make this explicit; otherwise consider introducing an opt-in flag in metadata
(e.g. persist: true) to allow persistence where desired.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cb232b9 and 3c535eb.

📒 Files selected for processing (2)
  • obsah/__init__.py (8 hunks)
  • tests/fixtures/playbooks/types/metadata.obsah.yaml (1 hunks)
🔇 Additional comments (1)
tests/fixtures/playbooks/types/metadata.obsah.yaml (1)

38-45: Good use of required_if, forbidden_if, and reset constraints

The conditional constraints and reset rule correctly capture the relationship between database_mode and database_host.
No further action required.

Comment on lines +24 to 31
database_mode:
help: Defaults to internal. Set to 'external' if using an external database.
choices:
- internal
- external
database_host:
help: Host of the PostgreSQL database.
constraints:
Copy link

@coderabbitai coderabbitai bot May 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

database_host is missing an explicit type declaration

All other variables in the same section specify a type (e.g. FQDN, File).
Without a type, downstream validation (data_types.register_types) will treat the value as an untyped string, potentially allowing invalid hosts (empty strings, IPv6 literals when only FQDNs are expected, etc.).

   database_host:
     help: Host of the PostgreSQL database.
+    type: FQDN

Please double-check that FQDN (or another suitable custom type) is available in data_types; otherwise the metadata parser will raise a validation error.


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Find the register_types function to locate where custom data types are defined
rg "def register_types" -n

# Search for any definition or registration of FQDN in the codebase
rg "FQDN" -n

Length of output: 287


Enforce FQDN validation for database_host

We’ve confirmed that FQDN is defined in obsah/data_types.py (and already used in tests), so an explicit type declaration will enforce correct host formats.

  • File: tests/fixtures/playbooks/types/metadata.obsah.yaml (lines 24–31)
  • Add the missing type: FQDN under database_host
   database_host:
     help: Host of the PostgreSQL database.
+    type: FQDN
   constraints:
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
database_mode:
help: Defaults to internal. Set to 'external' if using an external database.
choices:
- internal
- external
database_host:
help: Host of the PostgreSQL database.
constraints:
database_mode:
help: Defaults to internal. Set to 'external' if using an external database.
choices:
- internal
- external
database_host:
help: Host of the PostgreSQL database.
type: FQDN
constraints:
🤖 Prompt for AI Agents
In tests/fixtures/playbooks/types/metadata.obsah.yaml around lines 24 to 31, the
`database_host` entry is missing an explicit `type` declaration, which causes it
to be treated as an untyped string and may allow invalid values. To fix this,
add the line `type: FQDN` under `database_host` to enforce proper FQDN
validation, ensuring the value conforms to expected host formats.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It surprises me it comes up with this. It's not a bad suggestion, but I think FQDN doesn't allow IP addresses. @evgeni I don't think we have a host type, do we?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was surprised too, but it indeed makes sense.

We don't have a "Host" type yet. I wonder if we should force fqdn for ssl verification still. But here in the example we can go with fqdn anyway, as it's not used besides being an example

Comment on lines +400 to +421
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
try:
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg):
delattr(args, arg)
if args.obsah_reset:
for reset_arg in args.obsah_reset:
delattr(args, reset_arg)
except FileNotFoundError:
pass
return args
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

reset_args() can raise AttributeError when a key is absent

getattr(args, reset_key) and args.obsah_reset may fail if the attribute
was never set (e.g. a parameter wasn’t supplied on the CLI or via persistence).
Additionally, delattr() is called unconditionally and can throw if the
attribute has already been removed.

-                if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
+                if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key, None):
@@
-            if args.obsah_reset:
-                for reset_arg in args.obsah_reset:
-                    delattr(args, reset_arg)
+            for reset_arg in getattr(args, 'obsah_reset', []) or []:
+                if hasattr(args, reset_arg):
+                    delattr(args, reset_arg)

This guards against missing attributes and avoids unexpected crashes.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
try:
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg):
delattr(args, arg)
if args.obsah_reset:
for reset_arg in args.obsah_reset:
delattr(args, reset_arg)
except FileNotFoundError:
pass
return args
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
try:
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key, None):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg):
delattr(args, arg)
for reset_arg in getattr(args, 'obsah_reset', []) or []:
if hasattr(args, reset_arg):
delattr(args, reset_arg)
except FileNotFoundError:
pass
return args
🤖 Prompt for AI Agents
In obsah/__init__.py around lines 400 to 415, the reset_args function uses
getattr and delattr on args without checking if the attributes exist, which can
raise AttributeError if the attributes are missing. To fix this, use getattr
with a default value to safely access attributes and check if an attribute
exists with hasattr before calling delattr to avoid exceptions from missing
attributes.

@evgeni evgeni force-pushed the persist branch 2 times, most recently from 3af9391 to ddf1f78 Compare May 21, 2025 07:16
@evgeni
Copy link
Member Author

evgeni commented May 21, 2025

@coderabbitai review

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
obsah/__init__.py (1)

408-427: No mechanism to handle mutually exclusive parameters.

As mentioned in the PR objectives, there's no mechanism to automatically unset mutually exclusive parameters when one is set. While the reset rules in metadata help address this for some cases, it's still a limitation that should be documented.

Consider adding documentation for users explaining this limitation and how to properly manage parameters that could conflict with each other. Alternatively, enhancing the constraints system to explicitly model mutually exclusive parameters could be a future improvement.

Also applies to: 461-470

🧰 Tools
🪛 Ruff (0.11.9)

420-423: Use contextlib.suppress(AttributeError) instead of try-except-pass

Replace with contextlib.suppress(AttributeError)

(SIM105)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3c535eb and ddf1f78.

📒 Files selected for processing (2)
  • obsah/__init__.py (8 hunks)
  • tests/fixtures/playbooks/types/metadata.obsah.yaml (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/fixtures/playbooks/types/metadata.obsah.yaml
🧰 Additional context used
🪛 Ruff (0.11.9)
obsah/__init__.py

420-423: Use contextlib.suppress(AttributeError) instead of try-except-pass

Replace with contextlib.suppress(AttributeError)

(SIM105)

🔇 Additional comments (7)
obsah/__init__.py (7)

84-84: Good addition of reset rules in metadata.

The reset key in the metadata allows for defining conditional reset rules for parameters, helping to address the limitation of unsetting parameters when other parameters change.


246-246: Nice modern Python syntax with the walrus operator.

Using the walrus operator (:=) is a clean way to handle conditional assignment. This is a good update from the previous version.


250-272: Good implementation of parameter persistence configuration.

The static methods follow the established patterns in the class and are well-documented. The state path is set to /var/lib/obsah as discussed in the PR, which is appropriate for storing persistent state.


313-313: Good approach for tracking non-persistable parameters.

Using a set to track arguments that shouldn't be persisted is a clean approach. Adding common CLI arguments like 'playbook', 'verbose', 'extra_vars', 'action', and 'target' to this set is appropriate since these are typically context-specific to each invocation.

Also applies to: 320-321, 330-331, 336-336, 359-360, 380-380


344-351: Persisted values bypass argparse's type validation.

When loading persisted parameters from YAML, the native Python types are retained (e.g., integers stay as int). Because argparse validation/conversion only runs on raw string inputs, the types of persisted parameters aren't validated, which could lead to unexpected behavior.

Consider implementing one of these solutions:

  1. Convert values to strings before persisting:
-            yaml.safe_dump(persist_params, persist_file)
+            yaml.safe_dump(
+                {k: str(v) if not isinstance(v, str) else v
+                 for k, v in persist_params.items()},
+                persist_file,
+            )
  1. Re-parse persisted values through the parser before applying them
  2. Persist the exact CLI invocation instead of Python objects

373-378: Good solution for parameter reset functionality.

Adding --reset-<param> arguments for persistable parameters gives users explicit control over which parameters to reset. The distinction between positional and named arguments is also handled appropriately.


452-454: Reset logic is properly integrated in the main flow.

The reset_args function is appropriately called after parsing arguments but before validation, ensuring that parameters are reset as needed before proceeding.

Comment on lines +461 to +465
if application_config.persist_params():
persist_dir = os.path.dirname(application_config.persist_path())
if not os.path.exists(persist_dir):
os.makedirs(persist_dir, mode=0o770, exist_ok=True)
with open(application_config.persist_path(), 'w') as persist_file:
persist_params = dict(vars(args))
for item in parser.obsah_dont_persist:
persist_params.pop(item, None)
yaml.safe_dump(persist_params, persist_file)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Creating the persistence directory is a good practice.

The code properly creates the persistence directory if missing with appropriate permissions. However, as noted earlier, persisting native Python types can bypass argparse validation on reload.

Consider the same solutions proposed earlier for line 344-351 to ensure type validation happens consistently.


🏁 Script executed:

#!/bin/bash
# Check how persisted parameters are loaded and applied
grep -n "set_defaults.*persist_params" obsah/__init__.py

Length of output: 123


Ensure persisted params undergo Argparse type validation

The current persistence loader at line 349 uses:

subparser.set_defaults(**persist_params)

This bypasses any type= converters defined for arguments, so invalid or malformed persisted values won’t be caught. Please apply the same approach you used around lines 344–351 to enforce type checking on reload. For example, you can:

  • Build a CLI‐style argument list from persist_params (e.g. ["--foo", "123", "--bar", "baz"]) and pass it through subparser.parse_args(...) so that each value is validated and converted.
  • Or, for each key/value, look up the corresponding action’s type callable on the parser and explicitly cast before calling set_defaults.

This ensures that any bad data in your YAML file is rejected at startup rather than silently accepted.

🤖 Prompt for AI Agents
In obsah/__init__.py around lines 461 to 470, the persisted parameters are
loaded and set using set_defaults, which bypasses argparse's type validation. To
fix this, convert the persisted parameters into a CLI-style argument list (e.g.,
["--key", "value", ...]) and re-parse them with the appropriate parser's
parse_args method to enforce type checking and conversions. Replace the direct
set_defaults call with this parsing approach so that invalid or malformed
persisted values are caught during reload.

Comment on lines +408 to +422
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
try:
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg):
delattr(args, arg)
if args.obsah_reset:
for reset_arg in args.obsah_reset:
try:
delattr(args, reset_arg)
except AttributeError:
pass
except FileNotFoundError:
pass
return args

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

The reset_args function has potential AttributeError issues.

The function reset_args can raise AttributeError when accessing attributes that don't exist with getattr(args, reset_key) or when checking persist_params.get(reset_key) != getattr(args, reset_key).

Apply this improvement:

-                if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
+                if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key, None):

For the reset section, you're using try-except, but consider using contextlib.suppress as suggested by static analysis:

-            if args.obsah_reset:
-                for reset_arg in args.obsah_reset:
-                    try:
-                        delattr(args, reset_arg)
-                    except AttributeError:
-                        pass
+            import contextlib
+            for reset_arg in getattr(args, 'obsah_reset', []) or []:
+                with contextlib.suppress(AttributeError):
+                    delattr(args, reset_arg)

This handles missing attributes safely and improves code readability.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
try:
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg):
delattr(args, arg)
if args.obsah_reset:
for reset_arg in args.obsah_reset:
try:
delattr(args, reset_arg)
except AttributeError:
pass
except FileNotFoundError:
pass
return args
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
try:
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key, None):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg):
delattr(args, arg)
import contextlib
for reset_arg in getattr(args, 'obsah_reset', []) or []:
with contextlib.suppress(AttributeError):
delattr(args, reset_arg)
except FileNotFoundError:
pass
return args
🧰 Tools
🪛 Ruff (0.11.9)

420-423: Use contextlib.suppress(AttributeError) instead of try-except-pass

Replace with contextlib.suppress(AttributeError)

(SIM105)

🤖 Prompt for AI Agents
In obsah/__init__.py around lines 408 to 427, the reset_args function risks
raising AttributeError when using getattr(args, reset_key) if the attribute does
not exist. To fix this, use getattr with a default value to safely handle
missing attributes. Also, replace the try-except block around delattr calls with
contextlib.suppress(AttributeError) to cleanly suppress missing attribute errors
and improve readability.

@evgeni
Copy link
Member Author

evgeni commented May 21, 2025

@coderabbitai review

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
obsah/__init__.py (1)

344-351: 🛠️ Refactor suggestion

Persisted parameters bypass argparse type validation

When loading parameters from the YAML file, the code uses set_defaults() which bypasses argparse's type validation. This means invalid values could be loaded without proper conversion or validation.

Consider converting the persisted parameters to CLI-style arguments and re-parsing them:

        if application_config.persist_params():
            with contextlib.suppress(FileNotFoundError):
                with open(application_config.persist_path()) as persist_file:
                    persist_params = yaml.safe_load(persist_file)
                if persist_params:
-                    subparser.set_defaults(**persist_params)
+                    # Convert persisted params to CLI args to ensure type validation
+                    cli_args = []
+                    for key, value in persist_params.items():
+                        if value is not None:  # Skip None values
+                            cli_args.extend([f"--{key.replace('_', '-')}", str(value)])
+                    
+                    # Parse the CLI args to validate and convert types
+                    parsed_defaults, _ = subparser.parse_known_args(cli_args)
+                    subparser.set_defaults(**vars(parsed_defaults))
🧹 Nitpick comments (3)
obsah/__init__.py (3)

458-459: Consider more secure directory permissions

The current permissions (0o770) allow group members to write to the persistence directory. For sensitive parameters, a more restrictive mode like 0o700 might be more appropriate.

        if not os.path.exists(persist_dir):
-            os.makedirs(persist_dir, mode=0o770, exist_ok=True)
+            os.makedirs(persist_dir, mode=0o700, exist_ok=True)

372-378: Add a mechanism to reset all persisted parameters

The code provides a way to reset individual parameters but lacks a mechanism to reset all persisted parameters at once, which would be useful for troubleshooting.

Add a global reset parameter:

        if application_config.persist_params() and variable.parameter.startswith('--'):
            reset_param = variable.parameter.replace('--', '--reset-')
            subparser.add_argument(reset_param, help=f'Reset {variable.name}', action='append_const', dest='obsah_reset', const=variable.name)
        elif application_config.persist_params():
            # Don't persist positional arguments
            parser.obsah_dont_persist.add(variable.name)

+    # Add a global reset parameter to reset all persisted parameters
+    if application_config.persist_params():
+        subparser.add_argument('--reset-all', help='Reset all persisted parameters', action='store_true')

Then update the reset_args function to handle this case:

    if application_config.persist_params():
        args = reset_args(application_config, args.playbook.metadata, args)
+        # Handle --reset-all flag
+        if getattr(args, 'reset_all', False):
+            # Create an empty file to reset all parameters
+            with open(application_config.persist_path(), 'w') as persist_file:
+                yaml.safe_dump({}, persist_file)
+            # Reset the args object to remove persisted values
+            for attr in dir(args):
+                if not attr.startswith('_') and attr not in parser.obsah_dont_persist:
+                    with contextlib.suppress(AttributeError):
+                        delattr(args, attr)

456-465: Consider storing parameters per playbook

The current implementation stores all parameters in a single file, which could lead to parameter conflicts when different playbooks have parameters with the same name but different semantics.

Consider storing parameters per playbook:

    if application_config.persist_params():
        persist_dir = os.path.dirname(application_config.persist_path())
        if not os.path.exists(persist_dir):
            os.makedirs(persist_dir, mode=0o770, exist_ok=True)
-        with open(application_config.persist_path(), 'w') as persist_file:
+        # Use playbook-specific file
+        playbook_persist_path = os.path.join(persist_dir, f"{args.playbook.name}_parameters.yaml")
+        with open(playbook_persist_path, 'w') as persist_file:
            persist_params = dict(vars(args))
            for item in parser.obsah_dont_persist:
                persist_params.pop(item, None)
            yaml.safe_dump(persist_params, persist_file)

And update the loading code as well:

    @classmethod
    def persist_path(cls):
        """
        Where to persist parameters to
        """
-        return os.path.join(cls.state_path(), 'parameters.yaml')
+        return os.path.join(cls.state_path(), 'parameters.yaml')  # Default path
        if application_config.persist_params():
+            # Use playbook-specific file if it exists
+            playbook_persist_path = os.path.join(os.path.dirname(application_config.persist_path()), 
+                                               f"{playbook.name}_parameters.yaml")
+            persist_file_path = playbook_persist_path if os.path.exists(playbook_persist_path) else application_config.persist_path()
            with contextlib.suppress(FileNotFoundError):
-                with open(application_config.persist_path()) as persist_file:
+                with open(persist_file_path) as persist_file:
                    persist_params = yaml.safe_load(persist_file)
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ddf1f78 and 15680bc.

📒 Files selected for processing (2)
  • obsah/__init__.py (9 hunks)
  • tests/fixtures/playbooks/types/metadata.obsah.yaml (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/fixtures/playbooks/types/metadata.obsah.yaml
🔇 Additional comments (6)
obsah/__init__.py (6)

11-11: Good addition of contextlib import

The addition of the contextlib import correctly supports the use of contextlib.suppress() which is a more concise and Pythonic way of handling exceptions compared to try-except blocks.


85-85: Good addition of reset logic to metadata

The reset logic added to metadata correctly implements a mechanism to reset dependent parameters when specific parameters change, improving consistency.


247-249: Correct use of walrus operator

The change to use the walrus operator is a good modernization that makes the code more concise and readable.


251-273: Well-structured persistence configuration

The new static methods in ApplicationConfig are well-designed, providing clear configuration options for persistence behavior and file locations.


417-421: Good use of contextlib.suppress

Using contextlib.suppress for handling potential AttributeError exceptions is a more Pythonic approach than try-except blocks.


348-350: 🛠️ Refactor suggestion

Handle empty or invalid YAML files

The code doesn't handle the case where the YAML file exists but is empty or contains only comments, resulting in persist_params being None. This would cause an error in the if persist_params: check.

Ensure persist_params is a dictionary:

            with contextlib.suppress(FileNotFoundError):
                with open(application_config.persist_path()) as persist_file:
                    persist_params = yaml.safe_load(persist_file)
-                if persist_params:
+                if persist_params is not None and persist_params:

Likely an incorrect or invalid review comment.

Comment on lines +407 to +416
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
with contextlib.suppress(FileNotFoundError):
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg):
delattr(args, arg)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Address potential AttributeError in reset_args()

When comparing persisted values with current args, the code doesn't handle the case where an attribute might not exist in the args namespace. This can raise an AttributeError.

Use getattr() with a default value to prevent exceptions:

-                if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
+                if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key, None):
-                    for arg in reset_values:
-                        if arg in persist_params and persist_params[arg] == getattr(args, arg):
+                    for arg in reset_values:
+                        if arg in persist_params and persist_params[arg] == getattr(args, arg, None):
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
with contextlib.suppress(FileNotFoundError):
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg):
delattr(args, arg)
def reset_args(application_config: ApplicationConfig, metadata: dict, args: argparse.Namespace):
with contextlib.suppress(FileNotFoundError):
with open(application_config.persist_path()) as persist_file:
persist_params = yaml.safe_load(persist_file)
if persist_params:
for (reset_key, reset_values) in metadata['reset']:
if reset_key in persist_params and persist_params.get(reset_key) != getattr(args, reset_key, None):
for arg in reset_values:
if arg in persist_params and persist_params[arg] == getattr(args, arg, None):
delattr(args, arg)
🤖 Prompt for AI Agents
In obsah/__init__.py around lines 407 to 416, the reset_args function uses
getattr(args, reset_key) and getattr(args, arg) without a default value, which
can raise AttributeError if the attribute does not exist. Update these getattr
calls to include a default value (e.g., None) to safely handle missing
attributes and prevent exceptions during comparison.

@evgeni evgeni merged commit 88bc025 into master May 27, 2025
14 checks passed
@evgeni evgeni deleted the persist branch May 27, 2025 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants