Should we add the same to the Standalone section above?

Also we should update quickstart.html too.

Thanks for the comment @mimaison
I've moved this bullet further up where we include properties for both standalone and distributed.

Regarding the quickstart, the reason I didn't change this file is that it's not shown anymore. Which might be something to fix, but possibly not in this PR.

Thanks @kkonstantine.

Regarding the quickstart, it's currently not shown on the website, but this commit added it back. So once we sync the docs for 3.2, it should be back.

Yeah, I agree that it'd be good for this PR to add this information to the Connect section of the quickstart, since that's been added back for 3.2.

Thanks for the link @mimaison. In that case, it's worth updating the quick start here too. Let me do that

Previous list item paragraphs are on a single line, and it's probably useful to keep that formatting style. Also, a few nits:

I also have a few questions.

1. Should the path include the JAR file or the directory that contains the JAR file?
2. Would it be useful if we supplied an example plugin.path line to add?

I've built the distribution with this PR branch, and verified that the quickstart running the standalone worker fails if the file connectors are not in the plugin.path, but adding plugin.path=libs/connect-file-<version>.jar does work as long as the quickstart is run from the installation directory.

Thanks for the comment @rhauch
One note that I have is that I wouldn't recommend adding the relative path even if it works, because it possibly doesn't lead to robust deployments. plugin.path works when following symlinks so that can help.

plugin.path works with uber-jars but I wouldn't necessarily consider connect-file-<version>.jar an uber jar. I'd recommend the mainstream way which is to add the parent directory of any directories containing connector jars. Let me try to see how an example looks like here, because that's already just a bullet point.

Will users that are trying this for the first time understand what this means, or will this be enough of a barrier to cause them to abandon the quickstart? I'm not sure, but it might be worth to be more explicit here.

First, since this is a quickstart I think it's okay to duplicate some information here just so that users don't have to go elsewhere to learn what they need to do to run the quickstart.

Second, the referenced plugin.path configuration section says (emphasis mine):

The list should consist of top level directories that include any combination of ...

And, to run the quickstarts we want them to add the path to that JAR file to the plugin.path configuration property, not add the path to the lib/ directory that contains the connect-file-<version>.JAR file, as the referenced documentation suggests to do.

So, I suggest we be much more explicit here, and say something like:

We could make this a little easier if we changed the connect-standalone.properties and connect-distributed.properties files to add a few lines to the existing commented out section, so that all a user has to do is uncomment one line. For example:

...
# Set to a list of filesystem paths separated by commas (,) to enable class loading isolation for plugins
# (connectors, converters, transformations). The list should consist of top level directories that include 
# any combination of: 
# a) directories immediately containing jars with plugins and their dependencies
# b) uber-jars with plugins and their dependencies
# c) directories immediately containing the package directory structure of classes of plugins and their dependencies
# Note: symlinks will be followed to discover dependencies or plugins.
# Examples: 
# plugin.path=/usr/local/share/java,/usr/local/share/kafka/plugins,/opt/connectors,
#
# To run the quickstart that just uses the example file system connectors, uncomment this line:
# plugin.path=lib/connect-file-<version>.jar,
#
#plugin.path=

These are comments, and I think we can improve the comments without a KIP, especially since with this PR we're trying to limit the impact on the quickstarts.

I should add that I don't think adding the comments to the config properties is really all that necessary -- it might simplify things a bit, but at the same time we don't want users to do that in production.

Not including it in the configs just means the quickstart needs a bit more content to explain how the user should modify the config. Plus, maybe a note that says not to use these in production.

I agree. Better to add this example to the quick start itself rather than the config.

Again, my dilemma was regarding whether we should suggest a relative path + uber jar value which is not typical for production even on the quick start or direct users to the documentation of the property. I'm also a fan of quick starts that demonstrate a working example step-by-step. So, after your comment I've changed the quick start doc to contain an example. Let me know how it looks after that change when you get a chance.