Conversation
6e9da5d to
fd49c6b
Compare
dongsupark
left a comment
There was a problem hiding this comment.
First of all, thank you for huge effort you put into this PR.
I like the approach in general. I am still in middle of reading, but I wrote several comments below. Will read the rest tomorrow.
The thing is, if a PR has over 10 commits that have own different purposes, it becomes hard for reviewers to look into each commit, and follow what you want to do.
In general, I would recommend creating multiple PRs that have own purposes. For example one PR for trivial changes(renaming variables/functions, reordering, cleaning up), another PR for dealing with improving xml parts, and another PR for rewriting hasher structure like the beginning of this PR. Then you could get the easiest part merged immediately, and after that rebase harder PRs on top of that, and go through review cycles etc. That would be really helpful for reviewers.
Yes fair enough. I know that this PR seems almost random and I suppose it kind of is. I was just changing things as I thought of them, without small initial goals for refactoring -- so things ended up order of a more logical order. Will keep to smaller PRs in the future though. |
Since that is what structs implementing it do. This will make more sense with further commits. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
- Rather than provide Hashers (prev. HashAlgo) impl structs to users that can then use them in conjunction with the sha1/sha2 etc. hasher that they are meant to represent, allow the structs implementing Hasher (Sha1 and Sha2) to perform hashing logic themselves. Therefore add three new trait methods: new(), update(), and finalize(). - Remove Hasher::from_boxed(). There is no reason for it to exist if you encapsulate the hashing logic (update and finalize) into the trait itself. - Add associated trait method Hasher::try_from_hex_string() for use in the pre-existing sha256_hex mod (for use in hard_xml Reading). Note I rename this module to sha256_from_str since that it literally all it is for, and the word "hex" isn't used consistently across the module. - Add new sha1_from_str module that mimics the sha256 one. - Rather than having a Hash struct that has some methods, just define two type aliases Sha1Digest and Sha256Digest when the output of these hashing algorithms is needed (where you could otherwise use H::Output where H is a struct implementing Hasher. - Declare that Hasher::Output must implement PartialEq so that we can compare hash outputs (and therefor compare the above mentioned ShaXXXDigest types). - Simplify download.rs:hash_on_disk() with the fact that the trait implementations encapsulate their own hashing logic. - Temporarily switch to printing all instances of a Hasher::Output to a Debug print. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
And comment about that there is a better way to potentially do this in the future if a particular RFC stabilises. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
rustfmt almost always produces better formatting than doing it manually. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Planning on removing reliance on anyhow crate just for contexts. We can create custom error strings using a crate-specific Error enum. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
anyhow and digest Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
This was documenting as existing to provide serialisation of the underlying inner value of type uuid::Uuid with surrounding braces. Whenever uuids are used throughout the codebase I image they are used just as uuid::Uuids, therefore we don't need a type just for (de)seriailsation purposes -- just provide a `with = ` attribute for XmlRead and XmlWrite. Also provide here preliminary unit tests for reading/writing some omaha structs using the base uuid::Uuid type to prove this is sufficient. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
To be logically close to the types they are using. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
There were some structs that had custom implementation of XmlRead when it could have been derived, so derive instead. Since we cannot derive or manually implement XmlRead for url::Url, we must create a newtype Urls(Vec<Url>) and implement XmlRead for that (using the original code). Note that I am fairly sure that this doesn't change behaviour but I am yet to write unit tests. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
This also requires going through and deriving PartialEq on the response field structs. This is done behind a cfg_attr(test, ) though since there is currently no business-logic requirement for it to be derived fully. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
In a top-down way to make generated docs read better when they are written. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
It doesn't appear to have a need to exist; using a FileSize in places that could just be a usize doesn't provide any extra type-safety, and doesn't really provide any extra semantic meaning to a dev. Wherever it was used it was just being converted back and forth from a usize anyway. Furthermore, there is no special (de)serialisation logic that is required for it -- it is also just a usize in the XML. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
PR flatcar#87 created the download crate which caused merge conflicts with flatcar#88.
fd49c6b to
7d2e60e
Compare
PR flatcar#87 created the download crate which caused merge conflicts with flatcar#88. Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
7d2e60e to
2f69107
Compare
|
Just rebased this onto new |
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
b748c4d to
753c210
Compare
dongsupark
left a comment
There was a problem hiding this comment.
Our CI fails at clippy warnings by default. Can you please address them?
error: using `clone` on type `Option<[u8; 32]>` which implements the `Copy` trait
--> src/download/package.rs:104:13
|
104 | self.hash_sha256.clone(),
| ^^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `self.hash_sha256`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
= note: `-D clippy::clone-on-copy` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::clone_on_copy)]`
error: using `clone` on type `Option<[u8; 20]>` which implements the `Copy` trait
--> src/download/package.rs:105:13
|
105 | self.hash_sha1.clone(),
| ^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `self.hash_sha1`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 32]` which implements the `Copy` trait
--> src/download/package.rs:122:70
|
122 | debug!(" sha256 match? {}", self.hash_sha256 == Some(calculated_sha256.clone()));
| ^^^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `calculated_sha256`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 20]` which implements the `Copy` trait
--> src/download/package.rs:125:66
|
125 | debug!(" sha1 match? {}", self.hash_sha1 == Some(calculated_sha1.clone()));
| ^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `calculated_sha1`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 32]` which implements the `Copy` trait
--> src/download/package.rs:127:67
|
127 | ... self.hash_sha256 != Some(calculated_sha256.clone()) || self.hash_sha1.is_some() && self.hash_sha1 != Some(calculated_sha1.clone()) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `calculated_sha256`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 20]` which implements the `Copy` trait
--> src/download/package.rs:127:148
|
127 | ...) && self.hash_sha1 != Some(calculated_sha1.clone()) {
| ^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `calculated_sha1`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 32]` which implements the `Copy` trait
--> src/download/package.rs:155:34
|
155 | let hdhashvec: Vec<u8> = hdhash.clone().into();
| ^^^^^^^^^^^^^^ help: try removing the `clone` call: `hdhash`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 32]` which implements the `Copy` trait
--> src/download/mod.rs:117:65
|
117 | debug!(" sha256 match? {}", expected_sha256 == Some(calculated_sha256.clone()));
| ^^^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `calculated_sha256`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 20]` which implements the `Copy` trait
--> src/download/mod.rs:120:61
|
120 | debug!(" sha1 match? {}", expected_sha1 == Some(calculated_sha1.clone()));
| ^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `calculated_sha1`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 32]` which implements the `Copy` trait
--> src/download/mod.rs:122:61
|
122 | if expected_sha256.is_some() && expected_sha256 != Some(calculated_sha256.clone()) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `calculated_sha256`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `[u8; 20]` which implements the `Copy` trait
--> src/download/mod.rs:125:57
|
125 | if expected_sha1.is_some() && expected_sha1 != Some(calculated_sha1.clone()) {
| ^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `calculated_sha1`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `Option<[u8; 32]>` which implements the `Copy` trait
--> src/download/mod.rs:142:60
|
142 | || do_download_and_hash(client, url.clone(), path, expected_sha256.clone(), expected_sha1.clone()),
| ^^^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `expected_sha256`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
error: using `clone` on type `Option<[u8; 20]>` which implements the `Copy` trait
--> src/download/mod.rs:142:85
|
142 | || do_download_and_hash(client, url.clone(), path, expected_sha256.clone(), expected_sha1.clone()),
| ^^^^^^^^^^^^^^^^^^^^^ help: try removing the `clone` call: `expected_sha1`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
Pulls in flatcar/ue-rs#72, flatcar/ue-rs#74, flatcar/ue-rs#84, flatcar/ue-rs#85, flatcar/ue-rs#87, flatcar/ue-rs#88. Signed-off-by: Dongsu Park <dongsu@dpark.io>
Pulls in flatcar/ue-rs#72, flatcar/ue-rs#74, flatcar/ue-rs#84, flatcar/ue-rs#85, flatcar/ue-rs#87, flatcar/ue-rs#88. Signed-off-by: Dongsu Park <dongsu@dpark.io>
Pulls in flatcar/ue-rs#72, flatcar/ue-rs#84, flatcar/ue-rs#85, flatcar/ue-rs#87, flatcar/ue-rs#88. Signed-off-by: Dongsu Park <dongsu@dpark.io>
Pulls in flatcar/ue-rs#72, flatcar/ue-rs#84, flatcar/ue-rs#85, flatcar/ue-rs#87, flatcar/ue-rs#88, flatcar/ue-rs#90. Signed-off-by: Dongsu Park <dongsu@dpark.io>
Pulls in flatcar/ue-rs#72, flatcar/ue-rs#84, flatcar/ue-rs#85, flatcar/ue-rs#87, flatcar/ue-rs#88, flatcar/ue-rs#90. Signed-off-by: Dongsu Park <dongsu@dpark.io>
Pulls in flatcar/ue-rs#72, flatcar/ue-rs#84, flatcar/ue-rs#85, flatcar/ue-rs#87, flatcar/ue-rs#88, flatcar/ue-rs#90. Signed-off-by: Dongsu Park <dongsu@dpark.io>
dongsupark
left a comment
There was a problem hiding this comment.
Looks good. Thanks.
CI of flatcar/scripts#3310 passed, which means no regression with this PR.
Once you address clippy issues, I will merge it.
Yes I will do that later today, thanks |
Signed-off-by: james-parky <72101291+james-parky@users.noreply.github.com>
|
@dongsupark fixed clippy lint warnings |
Pulls in flatcar/ue-rs#72, flatcar/ue-rs#84, flatcar/ue-rs#85, flatcar/ue-rs#87, flatcar/ue-rs#88, flatcar/ue-rs#90. Signed-off-by: Dongsu Park <dongsu@dpark.io>
Pulls in flatcar/ue-rs#72, flatcar/ue-rs#84, flatcar/ue-rs#85, flatcar/ue-rs#87, flatcar/ue-rs#88, flatcar/ue-rs#90. Signed-off-by: Dongsu Park <dongsu@dpark.io>
General Improvements to the
omahaCrateThe following is a slightly ammended collection of the commit bodies from the constituent commits of this PR. They serve not only as a description of all changes, but also (if read in written order) may serve to explain the justification behind the changes.
Hashing Algorithms
HashAlgotrait toHashersince that is what structs implementing it do.Hashstruct andHashertrait. Rather than provideHasher(prev.HashAlgo) impl structs to users that can then use them in conjunction with the sha1/sha2 etc. hasher that they are meant to represent, allow the structs implementingHasher(Sha1 and Sha2) to perform hashing logic themselves. Therefore add three new trait methods:new(),update(), andfinalize(). Other methods can be added later if required, but for now this is sufficient.Hasher::from_boxed(). There is no reason for it to exist if you encapsulate the hashing logic (update and finalize) into the trait itself.Hasher::try_from_hex_string()for use in the pre-existingsha256_hexmod (for use inhard_xmlreading). Note I rename this module tosha256_from_strsince that it literally all it is for, and the word "hex" isn't used consistently across the module.sha1_from_strmodule that mimics the sha256 one. Move this and the other one tohash_types.rsto be defined together with the types they use.Hashstruct that has some methods, just define two type aliasesSha1DigestandSha256Digestwhen the output of these hashing algorithms is needed (where you could otherwise useH::OutputwhereHis a struct implementing Hasher.Hasher::Outputmust implementPartialEqso that we can compare hash outputs (and therefor compare the above mentionedShaXXXDigesttypes).download.rs:hash_on_disk()with the fact that the trait implementations encapsulate their own hashing logic.Hasher::Outputto aDebugprint.FINGERPRINT_SIZEtoHashertrait and comment about that there is a better way to potentially represent the output type of these structs in the future if a particular RFC stabilises.Requests and Responses
#[rustfmt::skip]s inresponse.rs.rustfmtalmost always produces better formatting than doing it manually.ErrorandResulttypes -- planning on removing reliance onanyhowcrate just forcontexts. We can create custom error strings using a crate-specificErrorenum. This also removes the need for thect-codecsdependency.anyhowanddigest.Uuidtype and provide unit tests for xml read/write. This was documenting as existing to provide serialisation of the underlying inner value of typeuuid::Uuidwith surrounding braces. Whenever uuids are used throughout the codebase I image they are used just asuuid::Uuids, therefore we don't need a type just for (de)seriailsation purposes -- just provide awith =attribute forXmlReadandXmlWriteimpls.omahastructs using the baseuuid::Uuidtype to prove this is sufficient.XmlErrorvalues.derive(XmlRead). There were some structs that had custom implementation ofXmlReadwhen it could have been derived, so derive instead. Since we cannot derive or manually implementXmlReadforurl::Url, we must create a newtypeUrls(Vec<Url>)and implementXmlReadfor that (based off the original code).XmlRead-ing newUrlsstruct.XmlRead-ingPackagestruct with a generic helper function. This also requires going through and derivingPartialEqon the response field structs. This is done behind acfg_attr(test, )though since there is currently no business-logic requirement for it to be derived fully.Other
type.rsand therefore theFileSizestruct. It doesn't appear to have a need to exist; using aFileSizein places that could just be ausizedoesn't provide any extra type-safety, and doesn't really provide any extra semantic meaning to a dev. Wherever it was used it was just being converted back and forth from ausizeanyway. Furthermore, there is no special (de)serialisation logic that is required for it -- it is also just ausizein the XML.Notes
request.rsandresponse.rsfiles will require conversation with someone more familiar with ue, but happy to do the work.