Skip to content

Introduce ConsoleLogRequestProcessor#6299

Merged
arvindsv merged 1 commit intogocd:masterfrom
arvindsv:allow_elastic_agents_to_write_to_job_console_log
Jun 5, 2019
Merged

Introduce ConsoleLogRequestProcessor#6299
arvindsv merged 1 commit intogocd:masterfrom
arvindsv:allow_elastic_agents_to_write_to_job_console_log

Conversation

@arvindsv
Copy link
Member

@arvindsv arvindsv commented May 17, 2019

Mainly for use by plugins to write information to the job console log. This is helpful to understand what's going on when an elastic agent is being brought up by a plugin.

Todo:

Testing / trying this out:

  1. You'll need code from this PR with something like this in your config:
Part of GoCD config
  <elastic>
    <agentProfiles>
      <agentProfile id="mydocker" clusterProfileId="localdocker">
        <property>
          <key>Hosts</key>
          <value />
        </property>
        <property>
          <key>Mounts</key>
          <value />
        </property>
        <property>
          <key>Command</key>
          <value />
        </property>
        <property>
          <key>Privileged</key>
          <value />
        </property>
        <property>
          <key>ReservedMemory</key>
          <value />
        </property>
        <property>
          <key>Environment</key>
          <value />
        </property>
        <property>
          <key>Cpus</key>
          <value />
        </property>
        <property>
          <key>Image</key>
          <value>gocd/gocd-agent-alpine-3.9:v19.3.0</value>
        </property>
        <property>
          <key>MaxMemory</key>
          <value />
        </property>
      </agentProfile>
    </agentProfiles>
    <clusterProfiles>
      <clusterProfile id="localdocker" pluginId="cd.go.contrib.elastic-agent.docker">
        <property>
          <key>go_server_url</key>
          <value>https://192.168.0.100:8154/go</value>
        </property>
        <property>
          <key>environment_variables</key>
          <value />
        </property>
        <property>
          <key>max_docker_containers</key>
          <value>1</value>
        </property>
        <property>
          <key>docker_uri</key>
          <value>unix:///var/run/docker.sock</value>
        </property>
        <property>
          <key>auto_register_timeout</key>
          <value>3</value>
        </property>
        <property>
          <key>docker_ca_cert</key>
          <value />
        </property>
        <property>
          <key>docker_client_cert</key>
          <value />
        </property>
        <property>
          <key>docker_client_key</key>
          <value />
        </property>
        <property>
          <key>private_registry_server</key>
          <value />
        </property>
        <property>
          <key>private_registry_username</key>
          <value />
        </property>
        <property>
          <key>private_registry_password</key>
          <encryptedValue />
        </property>
        <property>
          <key>enable_private_registry_authentication</key>
          <value>false</value>
        </property>
        <property>
          <key>private_registry_custom_credentials</key>
          <value>true</value>
        </property>
        <property>
          <key>pull_on_container_create</key>
          <value>false</value>
        </property>
      </clusterProfile>
    </clusterProfiles>
  </elastic>
  <pipelines group="defaultGroup">
    <pipeline name="Test">
      <materials>
        <git url="https://github.com/arvindsv/faketime.git" />
      </materials>
      <stage name="defaultStage">
        <jobs>
          <job name="defaultJob" elasticProfileId="mydocker">
            <tasks>
              <exec command="ls" />
            </tasks>
          </job>
          <job name="defaultJob2" elasticProfileId="mydocker">
            <tasks>
              <exec command="ls" />
            </tasks>
          </job>
        </jobs>
      </stage>
    </pipeline>
  </pipelines>
  1. A plugin which logs these messages will be needed. For now, Show plugin messages in job console log gocd-contrib/docker-elastic-agents-plugin#119 can be used.

How does it look?

Something like this:

2019_05_16_21-30-48

Review:

It's not a big change. ConsoleLogRequestProcessor is the main change. Everything else just supports it.

Questions people have asked:

  1. Should we have a time window in which the plugin is allowed to append to console log?
  2. Should we restrict which plugins can log?

My answer has been: Decided not to restrict. The plugins need to know the job identifier to write to their console log. It’s unlikely they’ll write to the wrong job, since they’ll be using a job identifier provided to them. If there’s a malicious plugin, then it has full access to the filesystem, you don’t really need this processor API to do anything.

@arvindsv
Copy link
Member Author

Marked as 19.5.0, because it's not necessary to be part of 19.4.0. /cc @maheshp

@arvindsv
Copy link
Member Author

arvindsv commented Jun 3, 2019

@GaneshSPatil @maheshp Should we merge this? By itself, it does nothing. Just provides the ability to write to a job's console log.

@maheshp
Copy link
Contributor

maheshp commented Jun 3, 2019

@arvindsv we should be able to merge it. Do you want somebody to review it, I haven't looked at it in detail though.

@arvindsv
Copy link
Member Author

arvindsv commented Jun 3, 2019

Yes, please. That would be good. Maybe @GaneshSPatil has seen it? He asked me some questions about it. I'll just request a review from him for now. We can change it later.

@arvindsv arvindsv requested a review from GaneshSPatil June 3, 2019 16:35
* For use by plugins to write information to the job console log.
@arvindsv arvindsv force-pushed the allow_elastic_agents_to_write_to_job_console_log branch from 7a79de4 to f9f6bd8 Compare June 4, 2019 15:30
@arvindsv arvindsv merged commit c66edbe into gocd:master Jun 5, 2019
private Integer pipelineCounter;

@Expose
private String pipelineLabel;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arvindsv -- Can you help me understand the need for pipelineLabel?
IMO, pipeline-name and pipeline-counter should be enough to identify pipeline instance.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GaneshSPatil You're right. Yesterday, when I was writing the documentation for it, I was wondering the same. I'll make a change and remove the need for it. Thank you!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I just added it as it was available in the JobIdentifier object. But, that's not a good reason. :)

Copy link
Member Author

@arvindsv arvindsv Jun 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, also: ConsoleService#appendToConsoleLog needs a full JobIdentifier.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in: #6397

I've verified that it works without the pipeline label.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks :)

arvindsv added a commit to arvindsv/gocd that referenced this pull request Jun 6, 2019
arvindsv added a commit that referenced this pull request Jun 6, 2019
…age-remove-pipeline-label

#6299 - Remove need for pipeline label in console log request.
arvindsv added a commit to arvindsv/gocd that referenced this pull request Sep 6, 2019
The only change from v1 is that it does not use camel case in its message.
arvindsv added a commit to arvindsv/gocd that referenced this pull request Sep 6, 2019
The only change from v1 is that it does not use camel case in its message.
arvindsv added a commit to arvindsv/gocd that referenced this pull request Sep 9, 2019
The only change from v1 is that it does not use camel case in its message.
GaneshSPatil pushed a commit that referenced this pull request Sep 19, 2019
The only change from v1 is that it does not use camel case in its message.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants