Enable ZooKeeper client to establish connection in read-only mode #4244

massakam · 2024-03-25T06:43:13Z

Motivation

If the system property readonlymode.enabled is set to true on a ZooKeeper server, read-only mode is enabled. Data can be read from the server in read-only mode even if that server is split from the quorum.
https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Experimental+Options%2FFeatures

To connect to the server in read-only mode, the client must also allow read-only mode. The ZooKeeperClient class in the bookkeeper repository also has an option called allowReadOnlyMode.

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperClient.java

Lines 219 to 222 in 15171e1

    
           public Builder allowReadOnlyMode(boolean allowReadOnlyMode) { 
        
               this.allowReadOnlyMode = allowReadOnlyMode; 
        
               return this; 
        
           }

However, even if this option is set to true, the connection to the server in read-only mode will actually fail. The cause is in the ZooKeeperWatcherBase class. When the ZooKeeperWatcherBase class receives the SyncConnected event, it releases clientConnectLatch and assumes that the connection is complete.

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperWatcherBase.java

Lines 128 to 144 in 15171e1

    
           switch (event.getState()) { 
        
           case SyncConnected: 
        
               LOG.info("ZooKeeper client is connected now."); 
        
               clientConnectLatch.countDown(); 
        
               break; 
        
           case Disconnected: 
        
               LOG.info("ZooKeeper client is disconnected from zookeeper now," 
        
                   + " but it is OK unless we received EXPIRED event."); 
        
               break; 
        
           case Expired: 
        
               clientConnectLatch = new CountDownLatch(1); 
        
               LOG.error("ZooKeeper client connection to the ZooKeeper server has expired!"); 
        
               break; 
        
           default: 
        
               // do nothing 
        
               break; 
        
           }

However, if the server is in read-only mode, it will receive ConnectedReadOnly instead of SyncConnected. This causes the connection to time out without being completed.

Changes

Modified the switch statement in the ZooKeeperWatcherBase class to release clientConnectLatch when ConnectedReadOnly is received if the allowReadOnlyMode option is true.

By the way, allowReadOnlyMode is never set to true in BookKeeper. So this change would be useless for BookKeeper. However, it is useful for Pulsar. Because Pulsar also uses ZooKeeperWatcherBase and needs to be able to connect to ZooKeeper in read-only mode.
https://github.com/apache/pulsar/blob/cba1600d0f6a82f1ea194f3214a80f283fe8dc27/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/PulsarZooKeeperClient.java#L242-L244

massakam · 2024-07-10T05:19:08Z

Ran git rebase master.

massakam · 2024-07-10T05:59:55Z

@shoothzj @dlg99 @hangc0276
All checks have passed so would you please merge this?

…ache#4244) ### Motivation If the system property `readonlymode.enabled` is set to true on a ZooKeeper server, read-only mode is enabled. Data can be read from the server in read-only mode even if that server is split from the quorum. https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Experimental+Options%2FFeatures To connect to the server in read-only mode, the client must also allow read-only mode. The `ZooKeeperClient` class in the bookkeeper repository also has an option called `allowReadOnlyMode`. https://github.com/apache/bookkeeper/blob/15171e1904f7196d8e9f4116ab2aecdf582e0032/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperClient.java#L219-L222 However, even if this option is set to true, the connection to the server in read-only mode will actually fail. The cause is in the `ZooKeeperWatcherBase` class. When the `ZooKeeperWatcherBase` class receives the `SyncConnected` event, it releases `clientConnectLatch` and assumes that the connection is complete. https://github.com/apache/bookkeeper/blob/15171e1904f7196d8e9f4116ab2aecdf582e0032/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperWatcherBase.java#L128-L144 However, if the server is in read-only mode, it will receive `ConnectedReadOnly` instead of `SyncConnected`. This causes the connection to time out without being completed. ### Changes Modified the switch statement in the `ZooKeeperWatcherBase` class to release `clientConnectLatch` when `ConnectedReadOnly` is received if the `allowReadOnlyMode` option is true. By the way, `allowReadOnlyMode` is never set to true in BookKeeper. So this change would be useless for BookKeeper. However, it is useful for Pulsar. Because Pulsar also uses `ZooKeeperWatcherBase` and needs to be able to connect to ZooKeeper in read-only mode. https://github.com/apache/pulsar/blob/cba1600d0f6a82f1ea194f3214a80f283fe8dc27/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/PulsarZooKeeperClient.java#L242-L244

) ### Motivation If the system property `readonlymode.enabled` is set to true on a ZooKeeper server, read-only mode is enabled. Data can be read from the server in read-only mode even if that server is split from the quorum. https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Experimental+Options%2FFeatures To connect to the server in read-only mode, the client must also allow read-only mode. The `ZooKeeperClient` class in the bookkeeper repository also has an option called `allowReadOnlyMode`. https://github.com/apache/bookkeeper/blob/15171e1904f7196d8e9f4116ab2aecdf582e0032/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperClient.java#L219-L222 However, even if this option is set to true, the connection to the server in read-only mode will actually fail. The cause is in the `ZooKeeperWatcherBase` class. When the `ZooKeeperWatcherBase` class receives the `SyncConnected` event, it releases `clientConnectLatch` and assumes that the connection is complete. https://github.com/apache/bookkeeper/blob/15171e1904f7196d8e9f4116ab2aecdf582e0032/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperWatcherBase.java#L128-L144 However, if the server is in read-only mode, it will receive `ConnectedReadOnly` instead of `SyncConnected`. This causes the connection to time out without being completed. ### Changes Modified the switch statement in the `ZooKeeperWatcherBase` class to release `clientConnectLatch` when `ConnectedReadOnly` is received if the `allowReadOnlyMode` option is true. By the way, `allowReadOnlyMode` is never set to true in BookKeeper. So this change would be useless for BookKeeper. However, it is useful for Pulsar. Because Pulsar also uses `ZooKeeperWatcherBase` and needs to be able to connect to ZooKeeper in read-only mode. https://github.com/apache/pulsar/blob/cba1600d0f6a82f1ea194f3214a80f283fe8dc27/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/PulsarZooKeeperClient.java#L242-L244 (cherry picked from commit 4d50a44)

apache/bookkeeper#4244

…ache#4244) ### Motivation If the system property `readonlymode.enabled` is set to true on a ZooKeeper server, read-only mode is enabled. Data can be read from the server in read-only mode even if that server is split from the quorum. https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Experimental+Options%2FFeatures To connect to the server in read-only mode, the client must also allow read-only mode. The `ZooKeeperClient` class in the bookkeeper repository also has an option called `allowReadOnlyMode`. https://github.com/apache/bookkeeper/blob/15171e1904f7196d8e9f4116ab2aecdf582e0032/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperClient.java#L219-L222 However, even if this option is set to true, the connection to the server in read-only mode will actually fail. The cause is in the `ZooKeeperWatcherBase` class. When the `ZooKeeperWatcherBase` class receives the `SyncConnected` event, it releases `clientConnectLatch` and assumes that the connection is complete. https://github.com/apache/bookkeeper/blob/15171e1904f7196d8e9f4116ab2aecdf582e0032/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperWatcherBase.java#L128-L144 However, if the server is in read-only mode, it will receive `ConnectedReadOnly` instead of `SyncConnected`. This causes the connection to time out without being completed. ### Changes Modified the switch statement in the `ZooKeeperWatcherBase` class to release `clientConnectLatch` when `ConnectedReadOnly` is received if the `allowReadOnlyMode` option is true. By the way, `allowReadOnlyMode` is never set to true in BookKeeper. So this change would be useless for BookKeeper. However, it is useful for Pulsar. Because Pulsar also uses `ZooKeeperWatcherBase` and needs to be able to connect to ZooKeeper in read-only mode. https://github.com/apache/pulsar/blob/cba1600d0f6a82f1ea194f3214a80f283fe8dc27/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/PulsarZooKeeperClient.java#L242-L244 (cherry picked from commit 4d50a44) (cherry picked from commit 1783677)

massakam force-pushed the allow-zk-ro-mode branch from 55dee7e to 7ecf92b Compare March 25, 2024 18:50

hezhangjian approved these changes May 12, 2024

View reviewed changes

hezhangjian requested review from StevenLuMT, dlg99, eolivelli, hangc0276, jiazhai, merlimat, wenbingshen and zymap May 12, 2024 09:08

dlg99 approved these changes May 30, 2024

View reviewed changes

hangc0276 approved these changes Jul 8, 2024

View reviewed changes

hangc0276 assigned massakam Jul 8, 2024

hangc0276 added release/4.15.6 release/4.16.7 release/4.17.2 area/bookie labels Jul 8, 2024

hangc0276 added this to the 4.18.0 milestone Jul 8, 2024

hangc0276 closed this Jul 8, 2024

hangc0276 reopened this Jul 8, 2024

massakam closed this Jul 10, 2024

massakam reopened this Jul 10, 2024

Enable ZooKeeper client to establish connection in read-only mode

26b8c9e

massakam force-pushed the allow-zk-ro-mode branch from 7ecf92b to 26b8c9e Compare July 10, 2024 05:16

hezhangjian merged commit 4d50a44 into apache:master Jul 10, 2024

massakam deleted the allow-zk-ro-mode branch July 10, 2024 06:12

lhotari added the cherry-picked/branch-4.17 label Apr 16, 2025

lhotari added the cherry-picked/branch-4.16 label Apr 16, 2025

lhotari added the cherry-picked/branch-4.15 label Apr 17, 2025

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 26, 2025

Enable ZooKeeper client to establish connection in read-only mode

7760d1d

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 26, 2025

Enable ZooKeeper client to establish connection in read-only mode

d04ea04

apache/bookkeeper#4244

StevenLuMT mentioned this pull request Jun 26, 2025

[improve][broker] Upgrade bookkeeper to 4.17.2/commons-configuration to 2.x/grpc to 1.72.0 and enable ZooKeeper client to establish connection in read-only mode apache/pulsar#24468

Merged

15 tasks

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 27, 2025

Enable ZooKeeper client to establish connection in read-only mode

c34ed14

apache/bookkeeper#4244

StevenLuMT mentioned this pull request Jun 27, 2025

[improve][broker] Upgrade bookkeeper to 4.17.2 StevenLuMT/pulsar#15

Open

15 tasks

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 27, 2025

Enable ZooKeeper client to establish connection in read-only mode

cbdc16b

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 28, 2025

Enable ZooKeeper client to establish connection in read-only mode

f23ed7e

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 28, 2025

Enable ZooKeeper client to establish connection in read-only mode

17cc165

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 28, 2025

Enable ZooKeeper client to establish connection in read-only mode

c79aef2

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 28, 2025

Enable ZooKeeper client to establish connection in read-only mode

64c60a4

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 30, 2025

Enable ZooKeeper client to establish connection in read-only mode

d9e981f

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 30, 2025

Enable ZooKeeper client to establish connection in read-only mode

c04db25

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jun 30, 2025

Enable ZooKeeper client to establish connection in read-only mode

00e1258

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jul 1, 2025

Enable ZooKeeper client to establish connection in read-only mode

f0a1d69

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jul 2, 2025

Enable ZooKeeper client to establish connection in read-only mode

1ee7fde

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jul 3, 2025

Enable ZooKeeper client to establish connection in read-only mode

97defb0

apache/bookkeeper#4244

StevenLuMT added a commit to StevenLuMT/pulsar that referenced this pull request Jul 4, 2025

Enable ZooKeeper client to establish connection in read-only mode

9535cec

apache/bookkeeper#4244

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable ZooKeeper client to establish connection in read-only mode #4244

Enable ZooKeeper client to establish connection in read-only mode #4244

Uh oh!

massakam commented Mar 25, 2024

Uh oh!

massakam commented Jul 10, 2024

Uh oh!

massakam commented Jul 10, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	public Builder allowReadOnlyMode(boolean allowReadOnlyMode) {
	this.allowReadOnlyMode = allowReadOnlyMode;
	return this;
	}

	switch (event.getState()) {
	case SyncConnected:
	LOG.info("ZooKeeper client is connected now.");
	clientConnectLatch.countDown();
	break;
	case Disconnected:
	LOG.info("ZooKeeper client is disconnected from zookeeper now,"
	+ " but it is OK unless we received EXPIRED event.");
	break;
	case Expired:
	clientConnectLatch = new CountDownLatch(1);
	LOG.error("ZooKeeper client connection to the ZooKeeper server has expired!");
	break;
	default:
	// do nothing
	break;
	}

Enable ZooKeeper client to establish connection in read-only mode #4244

Enable ZooKeeper client to establish connection in read-only mode #4244

Uh oh!

Conversation

massakam commented Mar 25, 2024

Motivation

Changes

Uh oh!

massakam commented Jul 10, 2024

Uh oh!

massakam commented Jul 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

massakam commented Jul 10, 2024 •

edited

Loading