New implementation of ssh-agent by Mi-La · Pull Request #2 · jenkinsci/ssh-agent-plugin

Mi-La · 2014-08-18T11:30:39Z

I have created a new ssh-agent implementation. It is related to post here:
https://wiki.jenkins-ci.org/display/JENKINS/SSH+Agent+Plugin?focusedCommentId=73533285&#comment-73533285
This new implementation uses native programs available on a jenkins slave machine.
i.e. If ssh-agent program is available in PATH on a slave, it is used and identities are added using ssh-add.

The current ssh-agent-plugin implementation chooses a first supported ssh agent implementation. It would be greate if a user could assign priorities to ssh agent implementations in the plugin cofiguration.

…stalled on a slave. Not working yet with keys protected by a passphrase.

… key.

cloudbees-pull-request-builder · 2014-08-18T11:32:36Z

plugins » ssh-agent-plugin #13 FAILURE
Looks like there's a problem with this pull request

Mi-La · 2014-08-18T11:42:39Z

Unfortunatelly I wasn't able to test it with the latest master. The last tested version is using pom.xml from commit SHA-1: 7926415 (* Oops didn't mean tp bump parent yet). With the latest master I'm only able to generate the .hpi package but a local jekins is not run properly.
It gives me following error:
WARNING: Failed to instantiate NodeMonitors
hudson.util.IOException2: Unable to read Z:\eccam\ssh-agent-plugin.\work\nodeMonitors.xml
at hudson.XmlFile.read(XmlFile.java:147)
at hudson.model.ComputerSet.(ComputerSet.java:410)
at hudson.model.OverallLoadStatistics.computeTotalExecutors(OverallLoadStatistics.java:63)
at hudson.model.LoadStatistics.updateExecutorCounts(LoadStatistics.java:188)
at hudson.model.LoadStatistics$LoadStatisticsUpdater.doRun(LoadStatistics.java:226)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: com.thoughtworks.xstream.converters.ConversionException: -1 : -1
---- Debugging information ----
message : -1
cause-exception : java.lang.ArrayIndexOutOfBoundsException
cause-message : -1
class : hudson.node_monitors.DiskSpaceMonitor
required-type : hudson.node_monitors.DiskSpaceMonitor
converter-type : hudson.util.RobustReflectionConverter
path : /hudson.util.DescribableList/hudson.node_monitors.DiskSpaceMonitor/ignored
line number : 10
class[1] : hudson.util.DescribableList
converter-type[1] : hudson.util.DescribableList$ConverterImpl

version : null

    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:79)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
    at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:71)
    at hudson.util.CopyOnWriteList$ConverterImpl.unmarshal(CopyOnWriteList.java:193)
    at hudson.util.DescribableList$ConverterImpl.unmarshal(DescribableList.java:263)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
    at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1061)
    at hudson.util.XStream2.unmarshal(XStream2.java:109)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1045)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.java:925)
    at hudson.XmlFile.read(XmlFile.java:143)
    ... 7 more

Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at com.thoughtworks.xstream.core.util.OrderRetainingMap.entrySet(OrderRetainingMap.java:77)
at java.util.HashMap.putMapEntries(HashMap.java:511)
at java.util.HashMap.putAll(HashMap.java:784)
at com.thoughtworks.xstream.core.util.OrderRetainingMap.(OrderRetainingMap.java:36)
at com.thoughtworks.xstream.converters.reflection.FieldDictionary.buildMap(FieldDictionary.java:135)
at com.thoughtworks.xstream.converters.reflection.FieldDictionary.fieldOrNull(FieldDictionary.java:113)
at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.getFieldOrNull(PureJavaReflectionProvider.java:186)
at hudson.util.RobustReflectionConverter.fieldDefinedInClass(RobustReflectionConverter.java:328)
at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:269)
at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:222)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
... 24 more

jenkinsadmin · 2014-08-18T12:01:52Z

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

Mi-La · 2014-09-09T11:54:50Z

I've found how to test it -> there was a problem with jenkins plugin version.
Needed to edit pom.xml:

org.jenkins-ci.plugins
plugin
1.557

With version 1.557 it works, probably related to bug JENKINS-19031

DiSB · 2014-10-24T22:46:55Z

Hi Mi-La,

The fix is still not working for me. I'm using Windows 2012. A SSH key with passphrase.
I downloaded and build the commit (ssh-agent-plugin-4de63017ab934857125ca6678219f480e43c2c44).
Here are the errors I get.

Building in workspace C:\Jenkins\jobs\dev1\workspace
[ssh-agent] Using credentials m.neag-a
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent] FATAL: Could not find a suitable ssh-agent provider
[ssh-agent] Diagnostic report
FATAL: [ssh-agent] Unable to start agent
hudson.util.IOException2: [ssh-agent] Unable to start agent
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.createSSHAgentEnvironment(SSHAgentBuildWrapper.java:231)
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:189)
at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:527)
at hudson.model.Run.execute(Run.java:1745)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:89)
at hudson.model.Executor.run(Executor.java:240)
Caused by: java.lang.RuntimeException: [ssh-agent] Could not find a suitable ssh-agent provider.
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper$SSHAgentEnvironment.(SSHAgentBuildWrapper.java:332)
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.createSSHAgentEnvironment(SSHAgentBuildWrapper.java:224)
... 7 more

Mi-La · 2014-10-27T09:11:14Z

Hi DiSB,
my agent implementation is not a patch for the current agents, it is an extension which allows to use native (already running on a system) ssh-agent. Therefor to make it working you will need to install ssh-agent on your Windows 2012 (and have path to it in the PATH environment variable).
My agent just tries to find ssh-agent binary and use it. ssh-agent is originally UNIX tool but it exists for Windows too (OpenSSH, Cygwin, MSYS MinGW).
The error you posted means that jenkins ssh-agent could not find any suitable ssh-agent implementation.
Installing ssh-agent on your Windows 2012 should help to make my implementation working.

nicolasr69 · 2015-02-06T15:22:26Z

Hi Mi-La,

I too have an issue while running Jenkins under windows server 2012 (No slaves).
I have installed the tomcat native libraries (http://tomcat.apache.org/native-doc/) and got the right tomcat logs messages:

6-feb-2015 15:46:46 org.apache.catalina.core.AprLifecycleListener init
INFO: Loaded APR based Apache Tomcat Native library 1.1.32.
6-feb-2015 15:46:46 org.apache.catalina.core.AprLifecycleListener init
INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
6-feb-2015 15:46:47 org.apache.coyote.http11.Http11AprProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8080

I also installed Cygwin (with OpenSSH) and Putty in this server.
But I still got this error on my Jenkins' job:

Started by user anonymous
Building in workspace E:\jenkins\jobs\Backup OVH\workspace
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent] FATAL: Could not find a suitable ssh-agent provider
[ssh-agent] Diagnostic report
FATAL: [ssh-agent] Unable to start agent
hudson.util.IOException2: [ssh-agent] Unable to start agent
    at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.createSSHAgentEnvironment(SSHAgentBuildWrapper.java:231)
    at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:189)
    at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:530)
    at hudson.model.Run.execute(Run.java:1718)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    at hudson.model.ResourceController.execute(ResourceController.java:89)
    at hudson.model.Executor.run(Executor.java:240)
Caused by: java.lang.RuntimeException: [ssh-agent] Could not find a suitable ssh-agent provider.
    at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper$SSHAgentEnvironment.<init>(SSHAgentBuildWrapper.java:332)
    at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.createSSHAgentEnvironment(SSHAgentBuildWrapper.java:224)
    ... 7 more

I use the latest version of the Jenkins' ssh-agent (1.5).

Putty is in the path, and I also copied pageant.exe as ssh-agent.exe and ssh-add.exe. Which is ridiculous but was my last hope !

Maybe I missed something when installing Cygwin or Putty... Could you tell me if I need to write something in the path ? or anything else ?

Please help !

I am using:

Tomcat 8.0.18
Java JDK 1.8.0_31
Jenkins 1.598

Mi-La · 2015-02-06T21:27:55Z

Hi nicolasr69,
did you build the Jenkins ssh-agent plugin with this patch? Please see my previou answer. My patch only adds a new Jenkins ssh-agent implementation which tries to find binary ssh-agent and ssh-add. I implemented it because I was facing similar problems as you describe ;-). The ssh-agent.exe from cygwin should work and putty shouldn't be needed. The path to ssh-agent.exe must be in PATH.
Hope it helps!

nicolasr69 · 2015-02-09T12:57:05Z

Hi Mi-La,

Thanks for your reactivity :)

I did not build the plugin, I installed it from the Jenkins' Plugins Management platform, after I installed (copied) the Tomcat Native Libraries.

I also forgot to add C:\cygwin\bin to the path... But I recently did it and it solved nothing. It can be good to know that Cygwin takes at least 3 minutes to start ! I don't know why... But can it be the reason that ssh-agent is marked as not found (if not responding) ?

I am trying to use MinGW instead. I added the path to MinGW's binaries to the Path and restarted Tomcat service. It still doesn't want to work ! :) However, in a shell, I can run ssh-agent bash and it opens a bash with ssh-agent running (ssh-add -L can connect to the ssh-agent).

I don't know what else I can do... I am also not used to Jenkins & Tomcat, but I don't think I did some big mistake...

Mi-La · 2015-02-09T13:08:29Z

Since this patch is not applied in standard jenkins ssh-agent plugin, it can work only if you build jenkins plugin by yourself.
You should checkout commit ssh-agent-plugin-4de63017ab934857125ca6678219f480e43c2c44.

nicolasr69 · 2015-02-11T15:49:01Z

Hi Mi-La !

Thanks for the comment I did not realize that your fork was not merged yet ! Sorry !

Anyway I tried to compile it with Maven but I did not get the ssh-agent.hpi that should result (am I right ?)
Here is the beginning of the error :

[WARNING] The POM for org.jenkins-ci.tools:maven-hpi-plugin:jar:1.106 is missing, no dependency information available
[WARNING] Failed to build parent project for org.jenkins-ci.plugins:ssh-agent:hpi:1.6-SNAPSHOT

I created a Gist here for the complete stderr output: https://gist.github.com/nicolasr69/9d59d22110affe5ada91

What is strange is that maven-hpi-plugin.pom does exist and is in the same folder ! (as maven-hpi-plugin.jar)

Do you have any idea ?

Mi-La · 2015-02-22T18:53:25Z

Hi nicolas,
yes, you should get ssh-agent.hpi. I also struggled with building, basically there were problems with pom.xml. I had to choice "compatible" versions of components. See one of my previous comments: #2 (comment).

Unfortunatelly I'm not able to post my built plugin (the ssh-agent.hpi) now. Let me know if it would help you to get "my" binary. Moreover the plugin still have some problems and it reports errors wrongly. But since it is working it is enough for me.

oleg-nenashev · 2015-07-17T03:48:50Z

Hello. Are there any updates/plans on this pull request? The change seems to be useful for me, but seems the pull request has not been finished

Mi-La · 2015-07-17T20:09:29Z

Hi, unfortunatelly I'm really busy with other things at the moment. If you know how to manage to get this pull request to be merged, please feel free to do it. If you need some interaction from me just let me know.
For me it was enough that it's working for me but I at least wanted to share it in case somebody would find it useful :-).

lkraider · 2015-09-29T02:01:13Z

+1
Please merge and release.

lkraider · 2015-09-29T21:39:49Z

src/main/java/com/cloudbees/jenkins/plugins/sshagent/exec/ExecRemoteAgentStarter.java

Changed this to extends MasterToSlaveCallable to build with latest master branch.

So what now? Could you do the change in a commit? Or shall I do it? Or do I need to do anything more? Thanks!

It's easier if you change in your repository and push to the branch you made the pull request from, then this pull request will get the new commit automatically.

lkraider · 2016-01-07T16:19:20Z

Running this on Mac OSX, working great.

azweb76 · 2016-02-12T15:08:18Z

SSH Agent is not adding keys on OSX. Will this fix the issue? If so, +1 on getting this merged in.

lkraider · 2016-07-06T16:20:21Z

@reviewbybees Please review this pull request for merging.

ghost · 2016-07-06T16:33:32Z

This pull request originates from a CloudBees employee. At CloudBees, we require that all pull requests be reviewed by other CloudBees employees before we seek to have the change accepted. If you want to learn more about our process please see this explanation.

srounce · 2016-08-02T08:09:47Z

What's the status on reviewing this?

stephenc · 2016-08-02T10:44:46Z

This looks good, but the callable needs to extend from MasterToSlaveCallable

kossmoboleat · 2016-11-02T15:30:19Z

Even when fixing the compile error this doesn't work on Windows.

kossmoboleat · 2016-11-03T13:34:54Z

I can confirm that it actually works if you have the correct executable on your PATH. With MSYS versions of ssh-agent, ssh-add and ssh everything worked. MSYS's commands are included in Git for Windows for example.

You have to add the bin-path to the global environment variables on the machine. I've only managed to get it to work with jenkins started directly with java -jar jenkins.war, which is called from the windows service.

I did have to extend the Starter with MasterToSlaveCallable, but then everything was fine.

jglick

Minor Remoting mistakes, but the idea looks sound.

jglick · 2016-11-05T21:07:02Z

src/main/java/com/cloudbees/jenkins/plugins/sshagent/exec/ExecRemoteAgentFactory.java

+    @Override
+    public boolean isSupported(Launcher launcher, final TaskListener listener) {
+        try {
+			Runtime.getRuntime().exec("ssh-agent -k");


This only tells you if ssh-agent exists on the master. You must use the provided Launcher.

janvrany · 2016-11-22T07:06:16Z

FYI: I have rebased Mi-La's changes onto master and it works like a charm for me. Thanks, guys!

You may find the code at: https://github.com/janvrany/ssh-agent-plugin/tree/pr2
I have not addressed (yet) jglick's comment about using provided Launcher.

Mi-La · 2016-11-22T07:22:28Z

Hi all, since I have no power to finish this pull request, I'm closing it right now. I believe that someone else will be able to finish it up correctly ;-).

lkraider · 2016-11-22T13:22:57Z

@janvrany can you open a PR with the updated code?

janvrany · 2016-11-22T20:19:40Z

@lkraider Sure, I will. I guess I need to address @jglick 's point first, though
I need to learn Jenkins API, never hacked it. And time is scarce here, too :-(

It would help if you could point me to an example of using Launcher and Proc...

lkraider · 2016-11-24T15:15:42Z

Hi @janvrany , I also have no real experience with that, but found this:
https://wiki.jenkins-ci.org/display/JENKINS/Hints+for+plugin-development+newbies

Example:

 launcher.launch("dir", new String[0], listener.getLogger(), build.getProject().getWorkspace());

This seems to now be deprecated in favor of using the ProcStarter, something like this should work for us (untested):

ProcStarter ps = launcher.launch();
ps = ps.cmds("ssh-agent -k").stdout(listener);
ps = ps.pwd(build.getWorkspace()).envs(build.getEnvironment(listener));
Proc proc = ps.start();

In fact, there is an execProcess method in this code, not sure why it isn't used here also, so just doing:

execProcess("ssh-agent -k");

should work too.

janvrany · 2016-11-24T15:27:06Z

@lkraider Perfect, thanks a lot! I'll try to find some time this weekend and have a look.

janvrany · 2016-11-25T23:31:09Z

@lkraider , all: I have just opened a new PR for updated code: New implementation of ssh-agent (second attempt). Let's move there.

Mi-La added 5 commits August 15, 2014 17:39

Adds a new remote agent implementation based on a native ssh-agent in…

5fe5918

…stalled on a slave. Not working yet with keys protected by a passphrase.

Support for keys protected by a passphrase.

9c4b8d9

Fixes executable flag in a temporary script for SSH_ASKPASS.

4865412

Modify copyright.

0a0f0d7

Fixes a deadlock when a wrong passphrase is associated with a private…

4de6301

… key.

Mi-La changed the title ~~Next implementation of ssh-agent~~ New implementation of ssh-agent Aug 18, 2014

lkraider reviewed Sep 29, 2015
View reviewed changes

jglick requested changes Nov 5, 2016

View reviewed changes

Mi-La closed this Nov 22, 2016

Mi-La reopened this Nov 22, 2016

Mi-La closed this Nov 22, 2016

janvrany mentioned this pull request Nov 25, 2016

New implementation of ssh-agent (second attempt) #17

Merged

Uh oh!

Conversation

Mi-La commented Aug 18, 2014

Uh oh!

cloudbees-pull-request-builder commented Aug 18, 2014

Uh oh!

Mi-La commented Aug 18, 2014

version : null

Uh oh!

jenkinsadmin commented Aug 18, 2014

Uh oh!

Mi-La commented Sep 9, 2014

Uh oh!

DiSB commented Oct 24, 2014

Uh oh!

Mi-La commented Oct 27, 2014

Uh oh!

nicolasr69 commented Feb 6, 2015

Uh oh!

Mi-La commented Feb 6, 2015

Uh oh!

nicolasr69 commented Feb 9, 2015

Uh oh!

Mi-La commented Feb 9, 2015

Uh oh!

nicolasr69 commented Feb 11, 2015

Uh oh!

Mi-La commented Feb 22, 2015

Uh oh!

oleg-nenashev commented Jul 17, 2015

Uh oh!

Mi-La commented Jul 17, 2015

Uh oh!

lkraider commented Sep 29, 2015

Uh oh!

lkraider Sep 29, 2015

Choose a reason for hiding this comment

Uh oh!

Mi-La Sep 30, 2015

Choose a reason for hiding this comment

Uh oh!

lkraider Sep 30, 2015

Choose a reason for hiding this comment

Uh oh!

lkraider commented Jan 7, 2016

Uh oh!

azweb76 commented Feb 12, 2016

Uh oh!

lkraider commented Jul 6, 2016

Uh oh!

ghost commented Jul 6, 2016

Uh oh!

srounce commented Aug 2, 2016

Uh oh!

stephenc commented Aug 2, 2016

Uh oh!

kossmoboleat commented Nov 2, 2016

Uh oh!

kossmoboleat commented Nov 3, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jglick left a comment

Choose a reason for hiding this comment

Uh oh!

jglick Nov 5, 2016

Choose a reason for hiding this comment

Uh oh!

janvrany commented Nov 22, 2016

Uh oh!

Mi-La commented Nov 22, 2016

Uh oh!

lkraider commented Nov 22, 2016

Uh oh!

janvrany commented Nov 22, 2016

Uh oh!

lkraider commented Nov 24, 2016

Uh oh!

janvrany commented Nov 24, 2016

Uh oh!

janvrany commented Nov 25, 2016

kossmoboleat commented Nov 3, 2016 •

edited

Loading