Skip to content

Conversation

@ryzhyk
Copy link
Contributor

@ryzhyk ryzhyk commented Feb 24, 2025

Which issue does this PR close?

Rationale for this change

When datafusion is used in a workspace that enables the rkyv-64 feature in the chrono crate, this triggered a Rust compilation error:

error[E0277]: can't compare `Option<&std::string::String>` with `Option<&mut std::string::String>`.

The root cause of the error is incorrect type unification in the Rust compiler, as explained in rkyv/rkyv#434.

What changes are included in this PR?

The workaround pushes the compiler in the right direction by converting the mutable reference to an immutable one manually.

Are these changes tested?

I don't think this requires additional tests.

Are there any user-facing changes?

no

When datafusion is used in a workspace that enables the `rkyv-64`
feature in the `chrono` crate, this triggered a Rust compilation error:

```
error[E0277]: can't compare `Option<&std::string::String>` with `Option<&mut std::string::String>`.
```

The root cause of the error is incorrect type unification in the Rust
compiler, as explained in rkyv/rkyv#434. The
workaround pushes the compiler in the right direction by converting the
mutable reference to an immutable one manually.

Signed-off-by: Leonid Ryzhyk <leonid@feldera.com>
@github-actions github-actions bot added the logical-expr Logical plan and expressions label Feb 24, 2025
Comment on lines +2874 to +2875
// this triggers a Rust compilation error:
// error[E0277]: can't compare `Option<&std::string::String>` with `Option<&mut std::string::String>`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to reproduce the error, but didn't:

fn main() {
    let mut s =  String::from("hello");
    let ss = String::from("hello");
    let option_ref: Option<&String> = Some(&ss);
    let mut_ref = &mut s;

    if option_ref == Some(mut_ref) //Option<&mut String> {
    {
        println!("They are equal!");
    }
}

Maybe I didn't notice anything?

Copy link
Contributor

@comphead comphead Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it compiler dependent? I was not able to reproduce either on latest compiler

UPD: i was able to reproduce it when enable feature in Cargo.toml

chrono = { version = "0.4.38", default-features = false, features = ["rkyv-64"] }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xudong963 , please modify your example like this to reproduce:

  • Add chrono as a dependency to Cargo.toml:
[dependencies]

chrono = { version = "0.4.38", default-features = false, features = ["rkyv-64"] }
  • Add chrono import to main.rs:
use chrono::*;

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ryzhyk -- seems reasonable to me.

A very strange bug

@comphead
Copy link
Contributor

I dont have a strong opinion here, we do not use rkyv-64 feature for chrono now so we shouldn't be fixing it imho. I'd rather file a bug to chrono to fix it on their side

@ryzhyk
Copy link
Contributor Author

ryzhyk commented Feb 25, 2025

I dont have a strong opinion here, we do not use rkyv-64 feature for chrono now so we shouldn't be fixing it imho. I'd rather file a bug to chrono to fix it on their side

The problem is Rust's feature unification: if this crate is used in a workspace that has at least one other crate that enables this chrono feature, it affects datafusion-expr, triggering the bug. Our project heavily relies on datafusion and without this fix it becomes unusable for us starting from version 45.0.0,

@ryzhyk
Copy link
Contributor Author

ryzhyk commented Feb 25, 2025

Thanks @ryzhyk -- seems reasonable to me.

A very strange bug

Indeed! It's triggered by something in rkyv, but it's really a compiler issue. However, it's unlikely to get fixed in the compiler, as (from what I understand) it's a heuristic that balances the depth of automatic dereferencing with compilation speed. The really unfortunate aspect of this is that through feature unification it can get triggered in unrelated crates like datafusion in this case.

@comphead
Copy link
Contributor

as long as it is documented I'm fine, sort of surprising why clippy allows extra dereferencing.

@comphead comphead merged commit 3a1d947 into apache:main Feb 25, 2025
24 checks passed
@alamb
Copy link
Contributor

alamb commented Feb 25, 2025

as long as it is documented I'm fine, sort of surprising why clippy allows extra dereferencing.

Yeah, me too -- but given the overhead of this workaround seems to have a low maintenance burden I think it is ok to include with DataFusion

Thanks for the contribution @ryzhyk

@ryzhyk
Copy link
Contributor Author

ryzhyk commented Feb 25, 2025

@alamb , @comphead , @xudong963 , thanks for the quick turnaround!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compilation error due to rkyv/rkyv#434

4 participants