memcached: Make config non-HA-aware (bsc#1038223)#1341
Closed
cmurphy wants to merge 1 commit intocrowbar:masterfrom
Closed
memcached: Make config non-HA-aware (bsc#1038223)#1341cmurphy wants to merge 1 commit intocrowbar:masterfrom
cmurphy wants to merge 1 commit intocrowbar:masterfrom
Conversation
Contributor
Author
|
Cloud7 version is here: #1340 (not cherry-picked) |
stefannica
previously approved these changes
Oct 2, 2017
nicolasbock
previously approved these changes
Oct 2, 2017
Contributor
|
NoMethodError: undefined method `get_memcached_servers' for MemcachedHelper:Module we need some other patch elsewhere? where is the code that sets up memchached replication? basically when starting the memcached we need to tell it which one is the replication master and how to reach it. |
Contributor
Author
|
@dirkmueller I had forgotten that the swift barclamp was still using that method name. I decided that since swift is special I would rather leave it alone, so I changed the method name back to plural and reverted the variable name changes, so this patch is a lot smaller now. |
Without this patch, when deployed in an HA configuration, all of the barclamps set their cache servers to all of the memcached servers in the cluster in lexicographical order. This is not actually an optimal way to configure memcached servers since if part of the cluster is down, the memcached servers living on it will be inaccessible. python-memcached is not tied to pacemaker and has no way of knowing that, so it passes the entire list to python-memcached which attempts to connect to each server serially, not attempting the next one until the first times out. The effect is that any query to the OpenStack service will take a very long time if the first memcached server in the list is down. This patch fixes the issue by only using the local memcached server instead of using all in the cluster. This is done for all barclamps using memcached except for swift, since swift has its own way of doing HA without pacemaker and also implements its own memcached client, so we might as well leave it alone.
Contributor
Author
|
Closing for the reasons given here: #1340 (comment) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Without this patch, when deployed in an HA configuration, all of the
barclamps set their cache servers to all of the memcached servers in the
cluster in lexicographical order. This is not actually an optimal way to
configure memcached servers since if part of the cluster is down, the
memcached servers living on it will be inaccessible. python-memcached is
not tied to pacemaker and has no way of knowing that, so it passes the
entire list to python-memcached which attempts to connect to each server
serially, not attempting the next one until the first times out. The
effect is that any query to the OpenStack service will take a very long
time if the first memcached server in the list is down.
This patch fixes the issue by only using the local memcached server
instead of using all in the cluster. This also adjusts the
get_memcached_servers helper method to only accept one node as input
since, knowing what we now know, we're unlikely to need more than one.
The get_memcached_servers method was implemented while updating the
barclamps to prevent deprecation warnings emitted by keystonemiddleware
in Ocata[1] and was mimicking old behavior used to set the cache servers
for keystone and nova.
[1] https://docs.openstack.org/releasenotes/keystonemiddleware/ocata.html