fix: extract converters from amends/extends base modules#52
Conversation
When a pkl file amends or extends a base module that defines output.renderer.converters, pklr was not extracting those converters because it only looked at the current module's body entries. This caused "missing field _type" deserialization errors in consumers like hk where Config.pkl (the base) defines the converters and hk.pkl amends it. Now extract converters from base module AST during amends/extends processing, before the current module's entries are evaluated. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical issue where Pkl converters defined in base modules were not being inherited by modules that Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces functionality to allow amending modules to inherit converters defined in their base modules' output blocks. The changes in src/eval.rs modify the Evaluator to extract converters from the 'output' property of base modules during evaluation, ensuring they are available for inherited objects. A new test case, converter_inherited_from_amends_base, has been added to tests/pkl_features.rs to validate this inheritance behavior. There is no feedback to provide on the review comments as none were supplied.
Greptile SummaryThis PR fixes a bug where Key changes and observations:
Confidence Score: 3/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant Caller
participant Evaluator
participant BaseModule
participant ChildModule
Caller->>Evaluator: eval_file(child.pkl, depth=0)
Evaluator->>ChildModule: parse(child.pkl)
note over Evaluator,ChildModule: child.pkl amends/extends Base.pkl
Evaluator->>BaseModule: eval_file(Base.pkl, depth=1)
BaseModule-->>Evaluator: Value::Object (base_obj)
note over Evaluator: Re-parse Base.pkl AST (depth=0, converters.is_empty())
Evaluator->>BaseModule: parse(Base.pkl)
loop base_module.body entries
alt Entry::ClassDef
Evaluator->>Evaluator: eval_class_def → scope
else Entry::Property("output") [NEW]
Evaluator->>Evaluator: extract_converters_from_ast → self.converters (base converters first)
end
end
note over Evaluator: Second pass – child module body
loop child_module.body entries
alt Entry::Property("output")
Evaluator->>Evaluator: extract_converters_from_ast → self.converters (child appended after base)
else Entry::Property(other)
Evaluator->>Evaluator: eval_expr → out map
end
end
Caller->>Evaluator: apply_converters(val)
note over Evaluator: Iterates self.converters in order — first match wins (base beats child if same class)
Evaluator-->>Caller: converted Value
|
| if let Entry::Property(prop) = entry | ||
| && prop.name == "output" | ||
| && depth == 0 | ||
| && self.converters.is_empty() | ||
| { | ||
| self.extract_converters_from_ast(prop, &scope, depth).await; | ||
| } | ||
| } |
There was a problem hiding this comment.
Base converters inserted before child converters, first-match wins
Converters are extracted from the base module here (amends processing, before the current module's body is evaluated). When the second pass later processes the child module's own output block (line ~658), any child-defined converters for the same class are appended after the base converters. Because apply_converters_recursive iterates the slice in order and returns on the first match (line ~2148), the base's converter takes precedence over the child's, which is the reverse of PKL's expected override semantics.
The fix applied to the extends branch has the same ordering issue (line ~571). Consider inserting base converters only when no child converter for that class already exists, or reversing the lookup order so child converters shadow base converters.
| let dir = std::env::temp_dir().join("pklr_test_amends_converter"); | ||
| let _ = std::fs::remove_dir_all(&dir); | ||
| std::fs::create_dir_all(&dir).unwrap(); | ||
|
|
||
| // Base module with class + converter |
There was a problem hiding this comment.
Fixed temp-dir path risks parallel-test collisions
The test writes to a hard-coded directory pklr_test_amends_converter inside temp_dir(). If cargo test runs this test concurrently with itself (e.g. two CI jobs sharing the same machine's /tmp), or if another test using the same name is added, the remove_dir_all / create_dir_all sequence can race and cause flaky failures.
Prefer a unique per-invocation path using tempfile::TempDir (already a common pattern in Rust test suites) or at minimum append the process ID:
let dir = std::env::temp_dir()
.join(format!("pklr_test_amends_converter_{}", std::process::id()));| assert_eq!(json["item"]["_type"], "foo"); | ||
| assert_eq!(json["item"]["x"], 42); | ||
| } | ||
|
|
||
| #[test] | ||
| fn converter_inherited_from_amends_base() { | ||
| use std::io::Write; | ||
| let dir = std::env::temp_dir().join("pklr_test_amends_converter"); | ||
| let _ = std::fs::remove_dir_all(&dir); | ||
| std::fs::create_dir_all(&dir).unwrap(); | ||
|
|
||
| // Base module with class + converter | ||
| let base_path = dir.join("Base.pkl"); | ||
| let mut base_file = std::fs::File::create(&base_path).unwrap(); | ||
| write!( | ||
| base_file, | ||
| r#" | ||
| class Step {{ | ||
| check: String = "" | ||
| }} | ||
|
|
||
| output {{ | ||
| renderer {{ | ||
| converters {{ | ||
| [Step] = (s) -> new Dynamic {{ | ||
| _type = "step" | ||
| ...s.toDynamic() | ||
| }} | ||
| }} | ||
| }} | ||
| }} | ||
| "# | ||
| ) | ||
| .unwrap(); | ||
|
|
||
| // Amending module | ||
| let child_path = dir.join("child.pkl"); | ||
| let mut child_file = std::fs::File::create(&child_path).unwrap(); | ||
| write!( | ||
| child_file, | ||
| r#"amends "Base.pkl" | ||
|
|
||
| myStep = new Step {{ | ||
| check = "cargo test" | ||
| }} | ||
| "# | ||
| ) | ||
| .unwrap(); | ||
|
|
||
| let rt = tokio::runtime::Runtime::new().unwrap(); | ||
| let json = rt.block_on(async { | ||
| let mut ev = Evaluator::new(); | ||
| let val = ev | ||
| .eval_source(&std::fs::read_to_string(&child_path).unwrap(), &child_path) | ||
| .await | ||
| .unwrap(); | ||
| let val = ev.apply_converters(val).await.unwrap(); | ||
| val.to_json() | ||
| }); | ||
| assert_eq!(json["myStep"]["_type"], "step"); | ||
| assert_eq!(json["myStep"]["check"], "cargo test"); | ||
| } |
There was a problem hiding this comment.
No test for the
extends code path
The PR modifies both the amends and extends branches of eval_module, but the new test only exercises the amends path. The file-based extends fix (lines ~566–573) and any future HTTP-extends fix go untested. Adding a second fixture using extends "Base.pkl" would give confidence that both branches behave correctly.
- Child converters now override base converters (clear before re-extract) - Remove is_empty() guard so base converters always populate as fallback - Use process ID in temp dir path to avoid parallel test collisions - Add converter_inherited_from_extends_base test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ffccf5e to
09c03c6
Compare
… generics
- Pass class name to eval_class_def and set type_name on ObjectSource
- Parse generic type params for Mapping/Map (previously skipped)
- Use the value type param as a default template for Mapping entries
- Preserve base ObjectSource in merge_values for type propagation
This enables converters to match objects created via typed Mappings
(e.g., `new Mapping<String, Step> { ["x"] { check = "echo" } }`).
Note: objects created via property amendment (e.g., `steps { ["x"] { ... } }`)
in amending modules do not yet carry type info, as pklr does not
implement type annotation propagation from class property declarations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bfa77dd to
80e6a53
Compare
Three interconnected fixes that enable converters to work with the
common pattern of bare objects in typed Mappings (e.g., hk.pkl's
`steps { ["echo"] { check = "echo ok" } }` amending a
`Mapping<String, Step>`):
1. Mapping generic type params: Parse generic params for Mapping/Map
(`new Mapping<String, Step> {}`) and inject a synthetic `default`
entry referencing the value type class. This makes the Mapping
carry type information for its values.
2. Body amendment scope: When a property has a body amendment
(`foo { ... }`) and the property exists in scope with ObjectSource,
use eval_amended_object instead of creating a fresh object. This
preserves the base value's type context through amendments.
3. Type-aware DynProperty evaluation: When a Mapping's default template
has ObjectSource, evaluate DynProperty values via eval_amended_object
rather than merge_values, and propagate the template's type_name to
the result. This enables converters to match nested typed objects.
Also seeds scope with base properties before processing child module
entries, so body amendments can find base values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
e0d95d5 to
42df72e
Compare
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## 🤖 New release
* `pklr`: 0.3.0 -> 0.4.0 (⚠ API breaking changes)
### ⚠ `pklr` breaking changes
```text
--- failure enum_tuple_variant_field_added: pub enum tuple variant field added ---
Description:
An enum's exhaustive tuple variant has a new field, which has to be included when constructing or matching on this variant.
ref: https://doc.rust-lang.org/reference/attributes/type_system.html#the-non_exhaustive-attribute
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/enum_tuple_variant_field_added.ron
Failed in:
field 2 of variant Expr::New in /tmp/.tmproz423/pklr/src/parser.rs:101
```
<details><summary><i><b>Changelog</b></i></summary><p>
<blockquote>
## [0.4.0](v0.3.0...v0.4.0) -
2026-03-25
### Fixed
- extract converters from amends/extends base modules
([#52](#52))
</blockquote>
</p></details>
---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).
Summary
Fixes the
missing field '_type'error in hk's CI (PR #769).PR #49 added
output.renderer.converterssupport, but only extracted converters from the current module's body entries. When a fileamendsorextendsa base module that defines converters (like hk'shk.pklamendingConfig.pkl), those converters were never extracted.converter_inherited_from_amends_basetest with two files on diskTest plan
converter_inherited_from_amends_base— creates Base.pkl with converters + child.pkl that amends it, verifies converters are appliedThis comment was generated by Claude Code.