Add docs for kops, explain when kops and when kubeadm by justinsb · Pull Request #1507 · kubernetes/website

justinsb · 2016-10-19T16:35:26Z

Quick getting started guide for kops.

Also try to provide some guidance as to when to use kops and when
kubeadm; based on discussions with sig-cluster-lifecycle that kubeadm is
a building block and not a provisioning tool.

I used "kubeadm" as a simplifying concept for "kubeadm and the other
work done by sig-cluster-lifecycle that kubeadm is a part of", in that
tools that don't (yet) use kubeadm are still leveraging the kubeadm
stream of work.

This change is

mikedanese · 2016-10-19T17:34:11Z

@kubernetes/sig-cluster-lifecycle

jbeda

I think it is great to document kops. I don't agree with many of the changes for how kubeadm is positioned here. I think the alpha label is enough to make people read the caveats.

jbeda · 2016-10-19T17:43:41Z

+but not under `example.com`).
+
+Let's assume you're using `dev.example.com` as your hosted zone.  You create that hosted zone using
+the [normal process](http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingNewSubdomain.html),


Can this be done by the AWS CLI? If so, that might be best as you assume that folks are using it in the next step.

People have trouble with this step, so it's much easier to point them to the official docs and get them into the control panel, IMO.

Is there value in providing both?

OK, seeing as you both want it, I'll add it :-)

jbeda · 2016-10-19T17:44:28Z

+that you have created, along with their configuration, the keys they are using etc.  This information is stored
+in an S3 bucket.  S3 permissions are used to control access to the bucket.
+
+Multiple clusters can use the same S3 bucket, and you can share an S3 bucket between your colleagues that


You should note the security implications of this. Any one with access to this state store will have "root" on your cluster.

jbeda · 2016-10-19T17:46:19Z

+
+Run "kops create cluster" to create your cluster configuration:
+
+`kops create cluster --zones=us-east-1c useast1.dev.example.com`


It is confusing that this doesn't actually start VMs. Might want to clarify that this is just configuration and doesn't actually start machines.

Good call - will fix

jbeda · 2016-10-19T17:47:34Z


 This quickstart shows you how to easily install a secure Kubernetes cluster on machines running Ubuntu 16.04 or CentOS 7.
-The installation uses a tool called `kubeadm` which is part of Kubernetes 1.4.
+The installation uses a tool called `kubeadm` which is alpha as of Kubernetes 1.4.


We have "alpha" in bold below. I don't think we need it here too.

Yeah, I was trying to side-step the issue of what does it mean for something to be "part of" kubernetes 1.4 vs not. I'll revert as this is a huge hairball...

jbeda · 2016-10-19T17:51:28Z

 It is simple enough that you can easily integrate its use into your own automation (Terraform, Chef, Puppet, etc).

-**The `kubeadm` tool is currently in alpha but please try it out and give us [feedback](/docs/getting-started-guides/kubeadm/#feedback)!**
+kubeadm is not a complete provisioning tool; it is designed as a part of a larger provisioning


"provisioning" is a really squishy word. I would say something like this and merge it with the paragraph above:

"""
kubeadm doesn't do all the work to create a cluster from scratch. Instead, it is assumed that the user already has a set of machines (virtual or real) that are up and running. A combination of native Linux distribution packages and container images are used to bootstrap Kubernetes on those machines. kubeadm is built to be able to integrate easily with other lower level automation (Terraform, Chef, Puppet, etc.).
"""

The problem I'm trying to solve is that users are genuinely confused about when they should use kubeadm, and when they should use systems like GKE or kops.

I don't mean to denigrate kubeadm, I think there's actually a clear distinction to be drawn:

If you want to build-your-own, use kubeadm and integrate with your systems

If you just want a working kubernetes cluster on GCE, and can live with the opinions, you should probably use GKE

If you just want a working kubernetes cluster on AWS, and can live with the opinions, you should probably use kops

If you can't live with either of these GKE/kops restrictions, you should use kubeadm.

I don't want to confuse people with details of the how at this point, so I merged our paragraphs to be:

kubeadm assumes you have a set of machines (virtual or real) that are up and running. It is designed to be part of a larger provisioning system - or just for easy manual provisioning. kubeadm is a great choice where you have your own infrastructure (e.g. bare metal), or where you have an existing orchestration system (e.g. Puppet) that you have to integrate with.

jbeda · 2016-10-19T17:52:41Z

+own infrastructure (e.g. bare metal), or where you have an existing orchestration system (e.g. Puppet)
+that you have to integrate with.
+
+If you are not constrained, other tools build on kubeadm to give you complete clusters:


I don't think we need to thread this through here. We have a navigation bar on the left that lists other choices. As things expand we won't want to update references all over the place and keep this stuff up to date. For example, if kops moves beyond just AWS, we'd have to come update this doc.

Sure, but we need a way to draw the distinction as users are confused.

jbeda · 2016-10-19T17:55:17Z

+* On AWS, [kops](kops) makes installation and cluster management easy (and supports high availability)
+
+**The `kubeadm` tool is currently in alpha but please try it out and give us [feedback](/docs/getting-started-guides/kubeadm/#feedback)!
+Be sure to read the [limitations](#limitations); in particular note that kubeadm is not currently recommended


I'm cool with pointing people at the limitation section below.

But I wouldn't say that it isn't recommended for GCE or AWS -- it is just more leg work to get it working. I'd rather document how to enable cloud providers than waive people off. We do have the --cloud-provider flag but haven't really documented how to use it well yet.

Again, the problem is that I'm seeing a lot of users with failed kubeadm installations on AWS, many of whom didn't understand that it wasn't sufficient to simply set the --cloudprovider flag.

Is there a way you can suggest to rephrase @jbeda ?

I'd rather fix up the documentation and teach people how to do AWS integration ("the hard way") rather than just waive them off. As time moves on we can build stuff in to kubeadm as needed and create clear separation.

If we don't have time to do this -- we can say something like "kubeadm doesn't have great support for automatically configuring cloud providers. Please refer to the specific cloud provider documentation or use another provisioning system".

I like that! And we can remove/repoint once those AWS/kubeadm docs are there.

jbeda · 2016-10-19T17:56:14Z

    <h3>Installing Kubernetes on Linux with kubeadm</h3>
-    <p>This quickstart will show you how to install a secure Kubernetes cluster on any computers running Linux, using a tool called <code>kubeadm</code> which is part of Kubernetes. It'll work with local VMs, physical servers and/or cloud servers, either manually or as part of your own automation. It is currently in alpha but please try it out and give us feedback!</p>
+    <p>This quickstart will show you how to install a secure Kubernetes cluster on any computers running Linux, using a tool called <code>kubeadm</code>. It'll work with local VMs, physical servers and/or cloud servers, either manually or as a part of your own automation. It is currently in alpha but please try it out and give us feedback!</p>
+    <p>If you are looking for a complete production cluster, note that kubeadm is intended as a building block.  Tools such as GKE and kops build on kubeadm to provision a complete cluster.</p>


I think that the "alpha" label does enough to warn folks agains production usage. And we don't want to brand kubeadm as a "toy".

My main change here was to add the second paragraph. I called it a "building block", not a "toy", which I believe is the correct positioning. I do not imply that kubeadm is unsafe for production. The intention is to say "kubeadm is a building block that you can build up into a complete solution, here are two opinionated solutions if you just want a k8s cluster now". Perhaps if I removed the word "production" from "complete production cluster" - but then "If you are looking for a complete cluster...." read awkwardly. Equally I felt "If you are looking for a production-ready cluster" would imply that kubeadm was not production-ready. Suggestions?

The reason the first paragraph came up in the diff is because I removed "which is part of Kubernetes", because

I didn't think it read very well

It certainly wouldn't have read very well if we then mentioned other tools. It would beg the question "Is GKE part of Kubernetes?" which is a real hairball ;-)

The page layout was awkward anyway. Personally I think we'll have to shorten this whole section anyway, but I thought this would be uncontroversial...

If you do care, I can put back "which is part of Kubernetes"

How about we replace "If you are looking for a complete production cluster" with "If you are looking for a fully automated solution" here?

Great suggestion! I like that way of describing it also "fully automated" because it is both non-judgemental, but clearly draws the distinction. And to some people "fully automated" is good, to some people it is bad, and the interpretation probably maps well to the direction people should go.

justinsb · 2016-10-20T04:57:31Z

Changes made per-review - thanks for the input @jbeda :-)

chrislovecnm

Looks great!

chrislovecnm · 2016-10-20T05:45:17Z

+
+## Feedback
+
+* Slack Channel: [#sig-aws](https://kubernetes.slack.com/messages/sig-aws/) has a lot of kops users


Do we want sig aws or cluster-lifecycle? I am thinking lifecycle.

cluster-lifecycle seems more fitting, but sig-aws really is where the activity is... either makes sense for now but that might change once kops is viable for other providers?

I think that makes sense. When/if we find that the kops discussion is primarily about kops & lifecycle, and not about kops & k8s & AWS then we can revisit.

jaygorrell · 2016-10-20T17:56:24Z

+If this is your first time using kops, do spend a few minutes to try those out!  An instance group is a
+set of instances, which will be registered as kubernetes nodes.  On AWS this is implemented via auto-scaling-groups.
+You can have several instance groups, for example if you wanted nodes that are a mix of spot and on-demand instances, or
+GPU and non-GPU instances.


I think it'd be helpful to cover the default instance sizing and the minimum recommended for masters and nodes. I know there are a lot of parameters covered in the kops docs but this is something that can cause a cluster to have problems right away.

I think we should point then to the kops docs for that. This is a general quick start guide. HA, sizing, spot instances - all that fun should be housed in the kops repo.

How does that sound?

@jaygorrell if the defaults don't work out of the box, do let me know as we should fix the defaults. I would like to keep the details out of this doc (and create a separate doc for those details - I feel we have reasonable docs on most kops features, but it is really hard to discover them). One thing that maybe does belong in this doc is things that can't be easily changed, but we don't have that doc yet anyway.

That's fair - I only mention it because of the suggestion that the reader spend some time playing with IGs, since the settings most people would play around with would involve cheap clusters to experiment. Leaving the details out here makes sense in general though of course.

It's a good point though... I added a paragraph that tries to provide some details by assimilation...

As an example: the nodes instance group defaults to t2.medium instances, but you might want to change this to an m4 or c4 class instance type for production usage, or you may want to use spot instances (by setting maxPrice).

(A snafu here is that us-east-2 doesn't support the m3.mediums we use elsewhere, so the defaults can get a little complicated...)

justinsb · 2016-10-22T00:30:22Z

Made the rest of the fixes per code review

justinsb · 2016-10-22T14:14:07Z

@jbeda are you OK with this now? You have a big red X by your review :-)

jbeda · 2016-10-22T21:53:51Z

I can live with this :) Thanks!

luxas · 2016-10-24T14:38:48Z

@mikedanese can merge this one when @justinsb has rebased

Adding the labels since it seems like everything's LGTM'd

justinsb · 2016-10-25T16:41:09Z

Rebased & squashed.

luxas · 2016-10-25T19:58:22Z

@mikedanese Any comments or can I merge this?

mikedanese · 2016-10-26T00:03:16Z

/lgtm

chrislovecnm · 2016-10-26T02:02:41Z

WTH does this still show as conflicting. This new feature from github is weird.

Quick getting started guide for kops. Also try to provide some guidance as to when to use kops and when kubeadm; based on discussions with sig-cluster-lifecycle that kubeadm is a building block and not a provisioning tool. I used "kubeadm" as a simplifying concept for "kubeadm and the other work done by sig-cluster-lifecycle that kubeadm is a part of", in that tools that don't (yet) use kubeadm are still leveraging the kubeadm stream of work.

googlebot added the cla: yes label Oct 19, 2016

k8s-github-robot assigned thockin Oct 19, 2016

thockin assigned mikedanese and unassigned thockin Oct 19, 2016

jbeda suggested changes Oct 19, 2016

View reviewed changes

chrislovecnm reviewed Oct 20, 2016

View reviewed changes

jaygorrell reviewed Oct 20, 2016

View reviewed changes

jbeda approved these changes Oct 22, 2016

View reviewed changes

luxas added Docs LGTM labels Oct 24, 2016

justinsb merged commit bfd03fe into kubernetes:master Oct 26, 2016


		Run "kops create cluster" to create your cluster configuration:

		`kops create cluster --zones=us-east-1c useast1.dev.example.com`


		## Feedback

		* Slack Channel: [#sig-aws](https://kubernetes.slack.com/messages/sig-aws/) has a lot of kops users

Conversation

justinsb commented Oct 19, 2016 • edited by thockin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mikedanese commented Oct 19, 2016

Uh oh!

jbeda left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

justinsb commented Oct 20, 2016

Uh oh!

chrislovecnm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

justinsb commented Oct 22, 2016

Uh oh!

justinsb commented Oct 22, 2016

Uh oh!

justinsb commented Oct 19, 2016 •

edited by thockin

Loading

jbeda commented Oct 22, 2016 •

edited

Loading