Skip to content

Interpolate shell inline config#8883

Merged
SwampDragons merged 11 commits intomasterfrom
fix_8835
Mar 13, 2020
Merged

Interpolate shell inline config#8883
SwampDragons merged 11 commits intomasterfrom
fix_8835

Conversation

@sylviamoss
Copy link
Copy Markdown
Contributor

@sylviamoss sylviamoss commented Mar 12, 2020

The shell provisioner wasn't receiving the generateData. This PR adds the generatedData to the shell provisioner context. Also, the file provisioner is now being interpolated as well with the generatedData and this was made for testing purposes.

template.json
Details
{
    "builders": [
        {
            "type": "amazon-ebs",
            "ami_name": "moss-packer-whee",
            "instance_type": "t2.micro",
            "source_ami_filter": {
              "filters": {
                "virtualization-type": "hvm",
                "name": "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*",
                "root-device-type": "ebs"
              },
              "owners": ["099720109477"],
              "most_recent": true
            },
            "ssh_username": "ubuntu"
        }
    ],
  "provisioners": [
    {
        "type": "shell-local",
        "inline": ["echo MOSS packer run uuid is '{{ build `PackerRunUUID`}}'"]
    },
    {
        "type": "shell",
        "inline": ["echo MOSS packer run uuid is '{{ build `PackerRunUUID`}}'"]
    }
  ]
}
output
Details
    amazon-ebs: MOSS packer run uuid is b846cf06-1f1f-4b8f-885b-a7fc89d7290e
2020/03/12 17:16:01 [INFO] (telemetry) ending shell-local
2020/03/12 17:16:01 [INFO] (telemetry) Starting provisioner shell
==> amazon-ebs: Provisioning with shell script: /var/folders/2b/xs01fttd7cbdwpzbgnd1k7380000gp/T/packer-shell382721493
2020/03/12 17:16:01 packer-provisioner-shell plugin: Opening /var/folders/2b/xs01fttd7cbdwpzbgnd1k7380000gp/T/packer-shell382721493 for reading
2020/03/12 17:16:01 packer-provisioner-shell plugin: [INFO] 81 bytes written for 'uploadData'
2020/03/12 17:16:01 [INFO] 81 bytes written for 'uploadData'
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] Opening new ssh session
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] Starting remote scp process:  scp -vt /tmp
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] Started SCP session, beginning transfers...
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] Copying input data into temporary file so we can read the length
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] scp: Uploading script_5242.sh: perms=C0644 size=81
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] SCP session complete, closing stdin pipe.
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] Waiting for SSH session to complete.
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] scp stderr (length 30): Sink: C0644 81 script_5242.sh
2020/03/12 17:16:01 packer-builder-amazon-ebs plugin: [DEBUG] Opening new ssh session
2020/03/12 17:16:02 packer-builder-amazon-ebs plugin: [DEBUG] starting remote command: chmod 0755 /tmp/script_5242.sh
2020/03/12 17:16:02 packer-builder-amazon-ebs plugin: [INFO] RPC endpoint: Communicator ended with: 0
2020/03/12 17:16:02 [INFO] RPC client: Communicator ended with: 0
2020/03/12 17:16:02 [INFO] RPC endpoint: Communicator ended with: 0
2020/03/12 17:16:02 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2020/03/12 17:16:02 packer-builder-amazon-ebs plugin: [DEBUG] Opening new ssh session
2020/03/12 17:16:02 packer-builder-amazon-ebs plugin: [DEBUG] starting remote command: chmod +x /tmp/script_5242.sh; PACKER_BUILDER_TYPE='amazon-ebs' PACKER_BUILD_NAME='amazon-ebs'  /tmp/script_5242.sh
2020/03/12 17:16:02 packer-builder-amazon-ebs plugin: [INFO] RPC endpoint: Communicator ended with: 0
2020/03/12 17:16:02 [INFO] 61 bytes written for 'stdout'
2020/03/12 17:16:02 [INFO] 0 bytes written for 'stderr'
2020/03/12 17:16:02 [INFO] RPC client: Communicator ended with: 0
2020/03/12 17:16:02 [INFO] RPC endpoint: Communicator ended with: 0
    amazon-ebs: MOSS packer run uuid is b846cf06-1f1f-4b8f-885b-a7fc89d7290e
acceptance test
Details The test runs only when PACKER_ACC=1
PACKER_ACC=1 go test ./provisioner/shell/provisioner_acc_test.go -v -timeout=45m
=== RUN   TestBuildShellProvisionerWithBuildVariablesSharing
2020/03/13 17:42:24 ui: amazon-ebs: output will be in this color.
2020/03/13 17:42:24 ui:
2020/03/13 17:42:24 Build debug mode: false
2020/03/13 17:42:24 Force build: false
2020/03/13 17:42:24 On error:
2020/03/13 17:42:24 Preparing build: amazon-ebs
2020/03/13 17:42:24 Waiting on builds to complete...
2020/03/13 17:42:24 Starting build run: amazon-ebs
2020/03/13 17:42:24 Running builder: amazon-ebs
...
go test ./provisioner/shell/provisioner_acc_test.go -v -timeout=45m
=== RUN   TestBuildShellProvisionerWithBuildVariablesSharing
--- SKIP: TestBuildShellProvisionerWithBuildVariablesSharing (0.00s)
    core.go:76: Acceptance tests skipped unless env 'PACKER_ACC' set
PASS
ok  	command-line-arguments	0.562s

closes #8835

@SwampDragons
Copy link
Copy Markdown
Contributor

Looks like this is also an issue for the powershell provisioner -- wanna tackle that one next?

@sylviamoss sylviamoss marked this pull request as ready for review March 13, 2020 17:24
@sylviamoss sylviamoss requested a review from a team as a code owner March 13, 2020 17:24
@sylviamoss sylviamoss added this to the 1.5.5 milestone Mar 13, 2020
writer := bufio.NewWriter(tf)
writer.WriteString(fmt.Sprintf("#!%s\n", p.config.InlineShebang))
for _, command := range p.config.Inline {
p.config.ctx.Data = generatedData
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not have this up by line 182 so we don't repeatedly re-set it for each line in inline?

@SwampDragons SwampDragons merged commit e94ff70 into master Mar 13, 2020
@SwampDragons SwampDragons deleted the fix_8835 branch March 13, 2020 19:51
nywilken added a commit that referenced this pull request Mar 27, 2020
…atching

Before change
```
413e19b Merge pull request #8942 from desolatorxxl/google-fix-ssh-keys-metadata
b81800d Merge pull request #8935 from zaventh/feature/start-on-boot
9486316 Merge pull request #8922 from hashicorp/f-vsphere_iso-export-ovf-options
56aebbe Merge pull request #8920 from rhencke/patch-1
d068430 make sure locals are evaluated only once variables are + test this (#8918)
3dae5df Merge pull request #8905 from hashicorp/fix_8493
811a730 Merge pull request #8907 from hashicorp/fix_8428
fa49d21 Merge pull request #8906 from hashicorp/fix_8904
23f5603 Merge pull request #8889 from hashicorp/hcl2_singular_blocks
dc9259f Merge pull request #8892 from zaventh/feature/vga-adapter
fc35f02 Merge pull request #8890 from hashicorp/fix_8880
7972ab7 Merge pull request #8735 from hashicorp/fix_plugin_loading
890d7b2 Merge pull request #8875 from hashicorp/fix_8812
e94ff70 Merge pull request #8883 from hashicorp/fix_8835
9075b80 Merge pull request #8891 from rhencke/patch-1
6477d8a Merge pull request #8882 from hashicorp/fix-var-file-hcl
6008f91 Merge pull request #8847 from takaishi/support-keyboard-interactive
5604561 Merge pull request #8877 from paulcichonski/remote-esxi-bastion
698f744 Merge pull request #8887 from hashicorp/untangle_ssh_docs_from_aws
aeedc9a Merge pull request #8879 from mbrancato/specify_keyvault_sku
5365fda Merge pull request #8884 from hashicorp/fix_codecov_config
4bd7b14 Merge pull request #8732 from jhawk28/reorder_cdrom_drive
072a71b Merge pull request #8863 from hashicorp/update_go-cty_regex
8a1caaa Merge pull request #8837 from hashicorp/fix_8730
7873cab Merge pull request #8858 from hashicorp/fix_8791
7e382d0 Merge pull request #8828 from mvitaly/fix_8816
8832b3e Merge pull request #8787 from jhawk28/vsphere_iso_multiple_disks
5281740 Merge pull request #8831 from rjhornsby/master
e35a872 Merge pull request #8830 from hashicorp/d-var-file-hcl2-not-yet
```

After change
```
⇶  git log v1.5.4...v1.5.5 --first-parent --oneline --grep="Merge pull request #[0-9]\+" --grep="(#[0-9]\+)$"
413e19b Merge pull request #8942 from desolatorxxl/google-fix-ssh-keys-metadata
c387dc2 builder/vsphere-clone: Find the vm within the folder (#8938)
b17b211 Add cleanup_remote_cache config option to vmware-iso (#8917)
e6368b9 Fix azure winrm_password attribution and allow to set winrm_username (#8928)
fcf10e9 Replace Amazon with Outscale for OSC BSU doc (#8944)
9240fb7 Fix typo in title (#8943)
2c6f096 Allow accepting image for the members in OpenStack builder (#8931)
b81800d Merge pull request #8935 from zaventh/feature/start-on-boot
daffd9c CONTRIBUTING: Update documentation for linting on Travis (#8933)
3a9d356 golangci-lint: Update --new-from-rev option to check only newly added commits (#8923)
97d797d Fix small typos in osc-bsuvolume.html.md (#8926)
9486316 Merge pull request #8922 from hashicorp/f-vsphere_iso-export-ovf-options
56aebbe Merge pull request #8920 from rhencke/patch-1
99b0b98 Add ovf export capability to vsphere builders (#8764)
d068430 make sure locals are evaluated only once variables are + test this (#8918)
ad8dafa HCL: add tests and fixes around var-file and var args  (#8914)
7979ab0 Add after_n_builds to codecov.yml (#8913)
3dae5df Merge pull request #8905 from hashicorp/fix_8493
811a730 Merge pull request #8907 from hashicorp/fix_8428
fa49d21 Merge pull request #8906 from hashicorp/fix_8904
b94937c Update provisioner_test.go (#8900)
2319521 Add iso config test for checksum from file specific case (#8897)
23f5603 Merge pull request #8889 from hashicorp/hcl2_singular_blocks
dc9259f Merge pull request #8892 from zaventh/feature/vga-adapter
690bf71 Add Codecov badge and remove report style (#8896)
fc35f02 Merge pull request #8890 from hashicorp/fix_8880
7972ab7 Merge pull request #8735 from hashicorp/fix_plugin_loading
890d7b2 Merge pull request #8875 from hashicorp/fix_8812
e94ff70 Merge pull request #8883 from hashicorp/fix_8835
```
@ghost
Copy link
Copy Markdown

ghost commented Apr 13, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Shell provisioner does not interpolate "inline" commands before writing them to file.

2 participants