Skip to content

Another attempt for the cache fix#6678

Merged
istio-testing merged 8 commits intoistio:masterfrom
costinm:costin-cachepush
Jun 28, 2018
Merged

Another attempt for the cache fix#6678
istio-testing merged 8 commits intoistio:masterfrom
costinm:costin-cachepush

Conversation

@costinm
Copy link
Copy Markdown
Contributor

@costinm costinm commented Jun 27, 2018

Fixes #5588 - the root problem was that events were squashed, but nothing triggered a push for the last events. So change #1 was pushed immediately, change #2 triggered a callback after (squash time).
Instead the callback is now after 1 second, and we evaluate if in the last second we received any event.
If not - push. Otherwise, keep waiting until the batch of changes settles down.

@costinm costinm requested review from andraxylia and hklai June 27, 2018 21:08
@codecov
Copy link
Copy Markdown

codecov bot commented Jun 27, 2018

Codecov Report

Merging #6678 into master will increase coverage by 1%.
The diff coverage is 0%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #6678    +/-   ##
=======================================
+ Coverage      70%     70%    +1%     
=======================================
  Files         351     351            
  Lines       30214   30107   -107     
=======================================
- Hits        21132   21070    -62     
+ Misses       8231    8195    -36     
+ Partials      851     842     -9
Impacted Files Coverage Δ
pilot/pkg/proxy/envoy/discovery.go 4% <0%> (ø) ⬇️
mixer/adapter/signalfx/metrics.go 41% <0%> (-2%) ⬇️
pilot/pkg/serviceregistry/kube/controller.go 66% <0%> (-2%) ⬇️
mixer/adapter/redisquota/redisquota.go 88% <0%> (-2%) ⬇️
mixer/adapter/rbac/rbac.go 10% <0%> (-1%) ⬇️
mixer/pkg/protobuf/yaml/resolver.go 59% <0%> (-1%) ⬇️
mixer/adapter/bypass/util.go 7% <0%> (ø) ⬇️
mixer/adapter/kubernetesenv/kubernetesenv.go 84% <0%> (ø) ⬇️
mixer/adapter/denier/denier.go 100% <0%> (ø) ⬆️
mixer/adapter/memquota/dedup.go 100% <0%> (ø) ⬆️
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a90f565...e964d05. Read the comment docs.

Copy link
Copy Markdown
Contributor

@ZackButcher ZackButcher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG, just a nit or two.

clearCacheMutex sync.Mutex
clearCacheTime = 1

clearCacheMutex sync.Mutex
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put the mutex at the top of this var block since it protects all of the variables here (mirroring the pattern that a mutex struct field should come before the other struct fields it guards).

})
}
// If last config change was > 1 second ago, push.
if time.Since(lastClearCacheEvent) > 1*time.Second {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How/why did we pick 1 second as the flush interval? Second, please make this a variable called something like maxCacheClearInterval and store it above with the cache vars.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point - I'll do both on follow-up PRs (since the bot and CI are taking many hours to run, and we're trying to get this into the monthly ) - I don't think either change are critical (or worse than the bug )

@ZackButcher
Copy link
Copy Markdown
Contributor

/lgtm

@istio-testing
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: costinm, ZackButcher

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@istio-testing
Copy link
Copy Markdown
Collaborator

istio-testing commented Jun 28, 2018

@costinm: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
prow/e2e-dashboard.sh e964d05 link /test e2e-dashboard
Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@istio-testing istio-testing merged commit a45f28b into istio:master Jun 28, 2018
quanjielin pushed a commit to quanjielin/istio that referenced this pull request Jul 2, 2018
* Improve cache push

* Format

* Improve push squashing (istio#6641)

* Improve cache push

* Format

* Revert experiment to verify if validation can be disabled
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants