feat(cli): Add scaffold command for destination#2055
feat(cli): Add scaffold command for destination#2055yevgenypats wants to merge 5 commits intomainfrom
Conversation
source will follow shortly. This will enable us to depreciate https://github.com/cloudquery/cq-provider-template and update v2 documentation accordingly
7792cd7 to
c8bc150
Compare
There was a problem hiding this comment.
Since scaffolding can turn into its own product, I tend not to put in the CLI in favor of a separate tool. We need to distinguish between users that consume the data (using the CLI) and plugin authors.
Also I think this is something we can do post v1 and remove the current docs about developing new plugins.
The current community plugins were developed tightly with the CloudQuery team anyways (correct me if I'm wrong), so maybe we can put this on hold for now pending a further discussion on how we want the plugin author experience to look like and also discuss our approach with the current authors.
Please let me know wdyt and maybe we can have a discussion with the rest of the team too
|
I Agree 100% that it might turn into it's own product but have the following caveats:
So to sum up I think we did enough improvement to the developer experience and we should invest more but rather just finalize it with this simple command and documentation and do it step by step rather then jumping straight ahead to full blown scaffold product when we dont have much contributors to external plugins. This will also give us a relatively easy way of getting operational experiance and feedback about scaffolding for a scaffold project if such will become a thing in the future. |
|
Thanks for the additional context. I should have been more clear with my suggestion. With all that being said, I'd rather see us ship fast and ship to learn, as we can always do a breaking change to remove commands if we'd like, just wanted to share my 2 cents. |
|
100% agree that in the docs we should say take a look at the gcp plugin, sdk codegen, talk to us on discord and so on. The working plugins are the most useful place to learn - no doubt about that. We removed the I do think we missed the very good point that you brought up about figuring the |
f3e970a to
e2b1c65
Compare
| } | ||
| } | ||
|
|
||
| func fetchSampleTable(ctx context.Context, meta schema.ClientMeta, parent *schema.Resource, res chan<- interface{}) error { |
There was a problem hiding this comment.
I think we should re-enforce the convention of placing fetch functions inside separate _fetch.go files here
| func (c *Client) Logger() *zerolog.Logger { | ||
| return &logger | ||
| } | ||
|
|
There was a problem hiding this comment.
I think the Client is missing a New function (referenced in main.go)
|
|
||
| import ( | ||
| "github.com/cloudquery/plugin-sdk/plugins" | ||
| "github.com/{{.Org}}/cq-source-{{.Name}}/client" |
There was a problem hiding this comment.
Missing github.com/cloudquery/plugin-sdk/schema import here
| @@ -0,0 +1,16 @@ | |||
| package client | |||
|
|
|||
There was a problem hiding this comment.
Missing imports section:
import (
"context"
"fmt"
"github.com/cloudquery/plugin-sdk/schema"
"github.com/cloudquery/plugin-sdk/specs"
"github.com/rs/zerolog"
)
|
I think I agree with @erezrokah that I'd rather wait for post-v1 with this functionality, but at the same time I do also agree that having something is better than having nothing. As long as we're okay with making breaking changes to this command (including maybe removing it down the line) I'm happy to go with it. That all said, the implementation looks like a great start; a couple of things I'll add:
|
Co-authored-by: Herman Schaaf <hermanschaaf@gmail.com>
Co-authored-by: Herman Schaaf <hermanschaaf@gmail.com>
Co-authored-by: Herman Schaaf <hermanschaaf@gmail.com>
|
ok I agree 🙃 . Let's skip this for now. |
source will follow shortly.
This will enable us to deprecate https://github.com/cloudquery/cq-provider-template and update v2 documentation accordingly