<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> **Engineering DRI:** @mattkasa ### Release post candidate Terraform modules are great to simplify, standardize and speed up infrastructure provisioning inside a company. Until now you could use only file or git based modules with GitLab or an external Terraform registry. By extending the GitLab registry with Terraform modules your modules can be found easily, moreover - as the registry supports semantic versioning - you get simple and secure module updates out of the box for the module's consumers. ### Problem to solve As a DevOps engineer, in order to use versioned modules with Terraform, I'd like to use a GitLab integrated Terraform registry. Currently, the only convenient way to store private Terraform providers is by using Terraform Cloud without having to write your own service. While this may work for some organizations, I would like to store custom in-house providers on my infrastructure. Since I already use GitLab extensively on my infrastructure, I would like to store the source code for the Terraform provider, the definition of the CI/CD pipeline, and the resulting packing all in the same place. ### Intended users Our own SREs building https://ops.gitlab.net/gitlab-com/gl-infra/terraform-modules * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) ### Further details There are many registries out there that we can learn from. Some that are related to infrastructure and configuration: - [Official Terraform Registry](https://registry.terraform.io/) - [Ansible Galaxy](https://galaxy.ansible.com/) ### Proposal - Under project settings I should be able to tell if my registry should be public or private. For public projects the registry is always public. (This is the same logic what we have with GitLab Pages.) - We want to support [semver versioning](https://semver.org/) in accessing modules.         ##### Iteration 1 * [ ] Gitlab supports the Registry protocol required by Terraform https://www.terraform.io/docs/internals/module-registry-protocol.html (gitlab#321102) * [ ] GitLab supports semantically versioned terraform modules (gitlab#321102) * [ ] Terraform can authenticate to Module registry (gitlab#324237) * Possible to use https://www.terraform.io/docs/commands/cli-config.html#credentials-1 ?/ * [ ] GitLab List Available Versions for a Specific Module (gitlab#321102) * :namespace/:name/:provider/versions * [ ] Terraform can Download Source Code for a Specific Module Version (gitlab#321102) * :namespace/:name/:provider/:system/:version/download * [ ] Add separate documentation page for the Terraform registry (https://gitlab.com/gitlab-org/gitlab/-/issues/327310) --- ##### Iteration 2 * [ ] Rollout feature flag https://gitlab.com/gitlab-org/gitlab/-/issues/325064 * [ ] Remove feature flag https://gitlab.com/gitlab-org/gitlab/-/issues/325065 --- ##### Iteration 3 * [ ] Add "Infrastructure Registry" to navigation * [ ] Workhorse? Terraform can discover the registry service https://www.terraform.io/docs/internals/remote-service-discovery.html#discovery-process #### How are modules published? We want to follow Terraform's requirements around modules. This means that a Terraform module can be any git repo with a `main.tf` file in its root. Moreover, the GitLab should support [the Terraform remote protocols specifications](https://gitlab.com/gitlab-org/gitlab/-/issues/220346). A module is published when: - a `main.tf` file is present in its root - a new semver compliant git tag is added to the repo In the MVC, we DON'T want to support sub-modules. #### Future considerations outside of initial scope - Terraform sub-module support - How can we help with module discovery? Once a module is published, it's useful only if people can find it. In the case of Terraform, best practice is to provide an example directory that shows the module in action (and is used for optional tests too). There are [tools](https://github.com/segmentio/terraform-docs) to generate automatic documentation for a module, do we want to fall back to it if no documentation exists? - An example repo with tests and following much practice setups: https://gitlab.com/sandlin/aws/terraform_eks/-/tree/master/ <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)?--> ### Documentation <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html Add all known Documentation Requirements here, per https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements If this feature requires changing permissions, this document https://docs.gitlab.com/ee/user/permissions.html must be updated accordingly. --> ### Availability & Testing <!-- What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### What does success look like, and how can we measure that? <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. --> ### What is the type of buyer? <!-- Which leads to: in which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers --> ### Links / references Some open source terraform registries: - https://github.com/apparentlymart/terraform-simple-registry - https://github.com/erikvanbrakel/anthology - https://github.com/outsideris/citizen