Skip to content

Reorganize comments in package templates#2869

Merged
jsoriano merged 5 commits intoelastic:mainfrom
jsoriano:fix-comments-in-templates
Sep 5, 2025
Merged

Reorganize comments in package templates#2869
jsoriano merged 5 commits intoelastic:mainfrom
jsoriano:fix-comments-in-templates

Conversation

@jsoriano
Copy link
Member

@jsoriano jsoriano commented Sep 3, 2025

Follow up to #2826 and #2707, fixes issues found in https://buildkite.com/elastic/integrations/builds/30790.

Reorganize templating in archetype templates so different delimiters are used depending on the stage. This helps rendering placeholders for the next step, and helps hiding comments that are not relevant in the files included in built packages.

The delimiters used are:

  • When creating packages, {[/]}.
  • When building packages, the default {{/}}.
  • Template comments {{/*/*/}} are used for comments that are intended for developers.
  • Markdown/HTML comments are used for comments that are expected on the final readmes.

In addition:

  • inputDocs includes the header, so it can be used to generate content only if there is any input.
  • Header about generated files is not included on build time, to avoid issues with existing packages (see https://buildkite.com/elastic/integrations/builds/30790).
  • A new function generatedHeader is added, that can be used in template docs files to include the header about generated files. It is added to README template for new packages.

@jsoriano jsoriano requested review from a team, haetamoudi, kgeller and mjwolf September 3, 2025 22:53
@jsoriano jsoriano self-assigned this Sep 3, 2025
@jsoriano
Copy link
Member Author

jsoriano commented Sep 3, 2025

test integrations

@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 3, 2025

⏳ Build in-progress, with failures

Failed CI Steps

History

cc @jsoriano

@elastic-vault-github-plugin-prod

Created or updated PR in integrations repository to test this version. Check elastic/integrations#15153

Copy link
Contributor

@mrodm mrodm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Just one question about a missing input in the generated README of a test package.

* [logfile](https://www.elastic.co/docs/reference/integrations/filestream)
* [apache/metrics](https://www.elastic.co/docs/reference/beats/metricbeat/metricbeat-metricset-apache-status)
<details>
<summary>httpjson</summary>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it is not related to the changes related to this PR, but is it expected that logfile input is missing here ?
Should it be added to internal/docs/_static/inputs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mjwolf @kgeller shall we add info for the missing inputs?

Btw, when we add or update input files the docs for packages using them will need an update and CI will fail, we will need to see how to handle this when we start modifying these files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we add info for the missing inputs?

yeah good catch, I think I got logfile confused with (deprecated) log when compiling the original list and we skipped it

@jsoriano did you want to add it here or should I open a separate PR?

Btw, when we add or update input files the docs for packages using them will need an update and CI will fail, we will need to see how to handle this when we start modifying these files.

right, this is something we didn't quite consider. we knew it would be difficult to update all packages, but not that it would temporarily break CI

@mjwolf maybe this is an issue docs phase 2 can also consider? it essentially relates back to the final bullet point Common documentation sections must be manually duplicated across integrations, making updates require mass manual changes across the entire portfolio.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we add info for the missing inputs?

yeah good catch, I think I got logfile confused with (deprecated) log when compiling the original list and we skipped it

@jsoriano did you want to add it here or should I open a separate PR?

Umm, it is linking to filestream, but it seems to be actually using the deprecated log input, so maybe we are fine if we don't want to document deprecated inputs?

Though I think there are many integrations still using this input.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we don't add it to the selections options when users are making a new datastream, but we still add it so existing integrations can have the documentation piece?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that would be a good option. Could you provide the files for logfile?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Will have something by the end of the week

"indent": indent,
"yamlString": packages.VarValueYamlString,
}
t := template.Must(template.New("template").Funcs(funcs).Delims("{[", "]}").Parse(templateBody))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to use different delimiters when creating and building packages 👍

@jsoriano
Copy link
Member Author

jsoriano commented Sep 5, 2025

I am going to merge this to fix the issues with existing packages.

@kgeller, please prepare the files for logfile in a follow up when possible, thanks!

@jsoriano jsoriano merged commit 0afb166 into elastic:main Sep 5, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants