Skip to content

e2e tests: use HaveField() for better error checking#14040

Merged
openshift-merge-robot merged 2 commits intocontainers:mainfrom
edsantiago:gomega_havefield
Apr 28, 2022
Merged

e2e tests: use HaveField() for better error checking#14040
openshift-merge-robot merged 2 commits intocontainers:mainfrom
edsantiago:gomega_havefield

Conversation

@edsantiago
Copy link
Copy Markdown
Member

This is a very late followup to my ginkgo-improving work of 2021.
It has been stuck since December because it requires gomega 1.17,
which we've just enabled.

This commit is simply a copy-paste of a command I saved in
my TODO list many months ago:

 sed -i -e 's/Expect(\([^ ]\+\)\.\([a-zA-Z0-9]\+\))\.To(Equal(/Expect(\1).To(HaveField(\"\2\", /' test/e2e/*_test.go

Signed-off-by: Ed Santiago santiago@redhat.com

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Apr 27, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: edsantiago

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 27, 2022
@edsantiago
Copy link
Copy Markdown
Member Author

Two commits, for ease of review (see commit message of 2nd commit for reason).

Tested by hand-mucking with init_test.go. Old results:

Expected
      <string>: initialized
  to equal
      <string>: initializing

New:

 Value for field 'Status' failed to satisfy matcher.      <<<<---- this is the diff
  Expected
      <string>: initialized
  to equal
      <string>: initializing

And, should it ever happen that the struct being tested does not even have an element with that name:

HaveField could not find field named 'Xtatus' in struct:
      <*define.InspectContainerState | 0xc0003d2000>: {
          OciVersion: "1.0.2-dev",
          Status: "initialized",
          Running: false,
          Paused: false,
          Restarting: false,
          OOMKilled: false,
          Dead: false,
          Pid: 1613918,
          ConmonPid: 1613915,
          ExitCode: 0,
          Error: "",
          StartedAt: 0001-01-01T00:00:00Z,
          FinishedAt: 0001-01-01T00:00:00Z,
          Health: {Status: "", FailingStreak: 0, Log: nil},
          Checkpointed: false,
          CgroupPath: "",
          CheckpointedAt: 0001-01-01T00:00:00Z,
          RestoredAt: 0001-01-01T00:00:00Z,
          CheckpointLog: "",
          CheckpointPath: "",
          RestoreLog: "",
          Restored: false,
      }

@edsantiago edsantiago force-pushed the gomega_havefield branch 2 times, most recently from 3a8f870 to 4018b9b Compare April 27, 2022 21:43
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really annoying:

  Value for field 'Value' failed to satisfy matcher.
  Expected
      <*string | 0xc000504380>: blue
  to equal
      <string>: blue

I've tried BeEquivalent() and BeEqual() and a variety of other dead ends. Does anyone know if there's an incantation for dereferencing Value, or for telling HaveField() to just shut up and Do What I Mean?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The below diff turned it green on my end:

diff --git a/test/e2e/generate_kube_test.go b/test/e2e/generate_kube_test.go                         
index f58b121f3070..31f2ea16e927 100644                                                              
--- a/test/e2e/generate_kube_test.go                                                                 
+++ b/test/e2e/generate_kube_test.go                                                                 
@@ -777,7 +777,8 @@ var _ = Describe("Podman generate kube", func() {                                
                Expect(pod.Spec.DNSConfig.Searches).To(ContainElement("foobar.com"))                 
                Expect(len(pod.Spec.DNSConfig.Options)).To(BeNumerically(">", 0))                    
                Expect(pod.Spec.DNSConfig.Options[0]).To(HaveField("Name", "color"))                 
-               Expect(pod.Spec.DNSConfig.Options[0]).To(HaveField("Value", "blue"))                 
+               s := "blue"                                                                          
+               Expect(pod.Spec.DNSConfig.Options[0]).To(HaveField("Value", &s))                     
        })                                                                                           
                                                                                                     
        It("podman generate kube multiple container dns servers and options are cumulative", func() {

Probably worth adding a helper function somewhere (utils package?) for such cases:

func StringToPointer(s string) *string {
     return &s
}

Copy link
Copy Markdown
Member

@vrothberg vrothberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM, really nice!

Copy link
Copy Markdown
Member

@Luap99 Luap99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but there is one downside with this function.

When I change the struct field the compiler/IDE would complain that the test is using a field which no longer exists. Now I will run the tests to get a failure much later.

This is a very late followup to my ginkgo-improving work of 2021.
It has been stuck since December because it requires gomega 1.17,
which we've just enabled.

This commit is simply a copy-paste of a command I saved in
my TODO list many months ago:

     sed -i -e 's/Expect(\([^ ]\+\)\.\([a-zA-Z0-9]\+\))\.To(Equal(/Expect(\1).To(HaveField(\"\2\", /' test/e2e/*_test.go

Signed-off-by: Ed Santiago <santiago@redhat.com>
Two for this error:

    invalid indirect of pod.Spec.DNSConfig.Options[0]

...and one for a gofmt error (spaces).

Signed-off-by: Ed Santiago <santiago@redhat.com>
@edsantiago
Copy link
Copy Markdown
Member Author

The Bad: new registry flake.

The Good: the only files touched in this PR are in test/e2e, so the following tests were all skipped: Compose, Compose v2, Unit, System, bud, upgrade, all with:

SKIPPING: This is a doc- and/or test-only PR with no changes under [whatever]

The Neutral: I have no opinion on the struct-change issue brought up by @Luap99, and will leave it for the team to weigh the tradeoffs. If struct renames are that common, please just close this without merging.

Thanks @vrothberg for the pointer fix!

@Luap99
Copy link
Copy Markdown
Member

Luap99 commented Apr 28, 2022

@edsantiago This not a problem for me I just wanted to point that out.
We should not rename the fields often. The error message is much more helpful this way so I like this.

@rhatdan
Copy link
Copy Markdown
Member

rhatdan commented Apr 28, 2022

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 28, 2022
@openshift-merge-robot openshift-merge-robot merged commit 87454cf into containers:main Apr 28, 2022
@edsantiago edsantiago deleted the gomega_havefield branch April 28, 2022 14:28
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants