Skip to content

refactor Value::follow_cell_path to reduce clones and return Cow#15640

Merged
fdncred merged 3 commits intonushell:mainfrom
Bahex:follow-cellpath-ref
May 1, 2025
Merged

refactor Value::follow_cell_path to reduce clones and return Cow#15640
fdncred merged 3 commits intonushell:mainfrom
Bahex:follow-cellpath-ref

Conversation

@Bahex
Copy link
Copy Markdown
Member

@Bahex Bahex commented Apr 25, 2025

Description

While working on something else, I noticed that Value::follow_cell_path receives self.

While it would be ideal for the signature to be (&'a self, cell_path) -> &'a Value, that's not possible because:

  1. Selecting a row from a list and field from a record can be done with a reference but selecting a column from a table requires creating a new list.
  2. Value::Custom returns new Values when indexed.

So the signature becomes (&'a self, cell_path) -> Cow<'a, Value>.

Another complication that arises is, once a new Value is created, and it is further indexed, the current variable

  1. can't be &'a Value, as the lifetime requirement means it can't refer to local variables
  2. shouldn't be Cow<'a, Value>, as once it becomes an owned value, it can't be borrowed ever again, as current is derived from its previous value in further iterations. So once it's owned, it can't be indexed by reference, leading to more clones

We need current to have two possible lifetimes

  1. 'out: references derived from &self
  2. 'local: references derived from an owned value stored in a local variable
enum MultiLife<'out, 'local, T>
where
    'out: 'local,
    T: ?Sized,
{
    Out(&'out T),
    Local(&'local T),
}

With current: MultiLife<'out, '_, Value>, we can traverse values with minimal clones, and we can transform it to Cow<'out, Value> easily (MultiLife::Out -> Cow::Borrowed, MultiLife::Local -> Cow::Owned) to return it

User-Facing Changes

Tests + Formatting

After Submitting

@github-actions github-actions bot added deprecated:pr-plugins Deprecated: use the A:plugins label instead A:LSP Language Server Protocol (nu-lsp) labels Apr 25, 2025
@Bahex Bahex added deprecated:optimization Deprecated: Use the performance label instead performance Work to make nushell quicker and use less resources and removed deprecated:pr-plugins Deprecated: use the A:plugins label instead A:LSP Language Server Protocol (nu-lsp) labels Apr 25, 2025
@cptpiepmatz
Copy link
Copy Markdown
Member

I see the usage of Cows but none of your MultiLife, did I miss something?

@Bahex
Copy link
Copy Markdown
Member Author

Bahex commented Apr 28, 2025

@cptpiepmatz

It's completely internal to follow_cell_path


fn get_value_member<'a>(
current: &'a Value,
member: &PathMember,
insensitive: bool,
) -> Result<ControlFlow<Span, Cow<'a, Value>>, ShellError> {

A single level member access is done with this function. It takes a reference, and if possible returns Cow::Borrowed, if not (selecting a column from a table) it returns Cow::Owned. (Don't worry about ControlFlow)


let mut store: Value = Value::test_nothing();
let mut current: MultiLife<'out, '_, Value> = MultiLife::Out(self);
for member in cell_path {
current = match current {
MultiLife::Out(current) => match get_value_member(current, member, insensitive)? {
ControlFlow::Break(span) => return Ok(Cow::Owned(Value::nothing(span))),
ControlFlow::Continue(x) => match x {
Cow::Borrowed(x) => MultiLife::Out(x),
Cow::Owned(x) => {
store = x;
MultiLife::Local(&store)
}
},
},
MultiLife::Local(current) => {
match get_value_member(current, member, insensitive)? {
ControlFlow::Break(span) => return Ok(Cow::Owned(Value::nothing(span))),
ControlFlow::Continue(x) => match x {
Cow::Borrowed(x) => MultiLife::Local(x),
Cow::Owned(x) => {
store = x;
MultiLife::Local(&store)
}
},
}
}
};
}

During traversal

  • We start with MultiLife::Out, which refers to &'out self.
  • At each step if get_value_member returns
    • Cow::Borrowed, we can continue using a reference with a lifetime of 'out, no cloning necessary.
    • Cow::Owned, getting the member required a clone, we can no longer continue using MultiLife::Out as it can't have a lifetime of 'out

I could have gone with just using Cow, but that would mean after we had to clone just once, we would have to continue using owned values.

  • Once we have an owned value we store it in store, and reference it with MultiLife::Local. So we can continue using references and avoid cloning further.

Ok(match current {
MultiLife::Out(x) => Cow::Borrowed(x),
MultiLife::Local(x) => {
let x = if std::ptr::eq(x, &store) {
store
} else {
x.clone()
};
Cow::Owned(x)
}
})

  • At the end of cell-path traversal, if we still have MultiLife::Out, we can return Cow::Borrowed, which means we avoided cloning at all!
  • If we have MultiLife::Local, it means we had to clone at some point and have a locally owned value.

    And there is one address comparison to see if we can just return store directly or have to clone the value we have.

@fdncred fdncred merged commit 55de232 into nushell:main May 1, 2025
16 checks passed
@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented May 1, 2025

Thanks

@github-actions github-actions bot added this to the v0.105.0 milestone May 1, 2025
@Bahex Bahex deleted the follow-cellpath-ref branch March 22, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deprecated:optimization Deprecated: Use the performance label instead performance Work to make nushell quicker and use less resources status:wait-until-after-nushell-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants