Skip to content

More interned string#5694

Merged
bors merged 9 commits intorust-lang:masterfrom
Eh2406:more_InternedString
Jul 8, 2018
Merged

More interned string#5694
bors merged 9 commits intorust-lang:masterfrom
Eh2406:more_InternedString

Conversation

@Eh2406
Copy link
Contributor

@Eh2406 Eh2406 commented Jul 6, 2018

This is a series of attempts to remove temporary strings. There is no evidence that this will help with anything, but it just seems cleaner.

Probably easier to read one commit at a time.

@rust-highfive
Copy link

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Borrow is implemented for InternedString would it allow lookups with &str here perhaps?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

impl Borrow<str> for InternedString Hmmm. We are currently Hashing InternedString and str the same. So we could. I have a vain hope that we can find some way to have O(1) hashing. but we don't now. So we can always remove Borrow if we gain faster hashing.

Another peace is to make a singleton for each constant string we interne, it is mostly just "default".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think for now having Borrow may be the best bet in that it'll hopefully give us a nice ergonomic boost in cases like this and also avoid unnecessary temporary allocations in theory

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, and it is an ergonomic boost. It should also save one call to hash and 2 cash misses. InternedString::new only allocates if the value is unseen, so this was not really allocating in the hot path. But optimizers are odd so maybe removing the unused branch will help.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for these the Cow was due to the derive above to handle escapes in names if they ever creep in

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So do you want me to leave the cows in, incase they are useful; or, do you want me to take them out and we can re-add them if needed?

Currently we are scrubbing data by reing the raw version from RegistryDependency then cleaning it into a local and using the local instead.

@Eh2406
Copy link
Contributor Author

Eh2406 commented Jul 7, 2018 via email

@Eh2406 Eh2406 force-pushed the more_InternedString branch from 63b59da to ea957da Compare July 7, 2018 15:13
@Eh2406
Copy link
Contributor Author

Eh2406 commented Jul 7, 2018

Cows are back, and test is added.

@alexcrichton
Copy link
Member

@bors: r+

👍

@bors
Copy link
Contributor

bors commented Jul 8, 2018

📌 Commit f4ef416 has been approved by alexcrichton

@bors
Copy link
Contributor

bors commented Jul 8, 2018

⌛ Testing commit f4ef416 with merge b02cd42...

bors added a commit that referenced this pull request Jul 8, 2018
More interned string

This is a series of attempts to remove temporary strings. There is no evidence that this will help with anything, but it just seems cleaner.

Probably easier to read one commit at a time.
@bors
Copy link
Contributor

bors commented Jul 8, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing b02cd42 to master...

@Eh2406
Copy link
Contributor Author

Eh2406 commented Jul 8, 2018

I was just looking at some profiles from >servo cargo generate-lockfile -Z no-index-update and RegistryPackage::Deserialize is ~3% So this clean up may have included some important parts of the code!

~90% was checking if files exist, reading files, or asking git to do the same.

@alexcrichton
Copy link
Member

Nice! How long does that take locally for you as well? 90% checking for files and such is pretty extreme, and it may be the next scaling bottleneck for Cargo...

@Eh2406 Eh2406 deleted the more_InternedString branch July 8, 2018 16:12
@Eh2406
Copy link
Contributor Author

Eh2406 commented Jul 8, 2018

~6 sec. I am a windows based lifeform and my AV may be uppity about large amounts of fs activity.

@alexcrichton
Copy link
Member

Oh jeez yeah, 6s just to generate a lockfile is pretty huge! That's a cost that's always paid on startup for Cargo :(

@Eh2406
Copy link
Contributor Author

Eh2406 commented Jul 8, 2018

I think it is not so long for evry cargo command. I don't wait that long for other projects. I think it is just that servo has a lot of git dependencies specified. Eche one needs to be checked out and parsed and checked for being a workspace, each time it is updated. So I don't think that a build command would have to do as much work, not that I tested it yet. I am sure, if everything was that slow we would hear it from the servo team. :-)

servo>cargo +nightly generate-lockfile -Z no-index-update
    Updating git repository `https://github.com/servo/osmesa-src`
    Updating git repository `https://github.com/servo/webrender`
    Updating git repository `https://github.com/servo/gaol`
    Updating git repository `https://github.com/servo/devices`
    Updating git repository `https://github.com/energymon/energymon-rust.git`
    Updating git repository `https://github.com/pcwalton/signpost.git`
    Updating git repository `https://github.com/servo/rust-azure`
    Updating git repository `https://github.com/servo/fontsan`
    Updating git repository `https://github.com/energymon/energymon-sys.git`

@alexcrichton
Copy link
Member

Ah ok that makes more sense!

@ehuss ehuss added this to the 1.29.0 milestone Feb 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants