Adding and removing submodule from git branch

I am constantly coming back and forwards with this – my current project requires to work with submodules, which i think is a great idea when you work in distributed devops environment and you want to have better control over the infrastructure-as-code configuration. It is just a quite complex process to commit submodules to the main stream.

Most of the time I am getting into merging issues every time i do the update. Therefore for my own benefit and hopefully others in the future, I decided to list steps I follow to remove and add the submodule in a simple way:

1. Create feature branch from develop branch of the main repository that holds submodule reference.

Note: work only on feature branch till you 100% you have all latest submodule SHA in place.

2. Clone your feature branch without recursive flag, i.e

git clone  -b feature/myNewUpdate git@<IP of the branch>:<repo groupID>/<repo_name>.git

(you will have to have all git config set beforehand with ssh keys etc.)

This will download the branch but will not load any submodule files with it with is what we are looking for.

3. Remove the submodule:

  • Delete submodule definition from the .gitmodules file.
  • Add changes by executing: git add .gitmodules
  • Delete section from .git/config.
  • Execute git rm --cached <submodule directory> (no trailing slash).
  • Run rm -Rf .git/modules/<submodule directory>
  • Commit changes git commit -m "Removed submodule <name>"
  • Delete any leftovers files
    rm -rf <submodule directory>
  • push changes if this is only what you intended to do – if you want to add submodule in the same go follow steps below:
  • git submodule add git@<IP>:<repo> <new submodule directory>
  • git submodule update –init <new submodule directory>
  • git submodule update –remote –merge  <new submodule directory>
  • git add  <new submodule directory>
  • git commit  <new submodule directory>
  • git push

This above should remove and add new submodule reference to your feature branch. Test it all works by going to the repository and navigate to your submodule and see if all the latest changes match – this will also reflect in updated @SHA next to submodule directory.

The last steps is to request the merge to main branch once all is tested in your feature branch.

Good luck!


Jenkins – dynamic choice parameter from Nexus repository

Embarrassingly, it took me couple of days to get it sorted but finally managed to get what i wanted.

Lets start from the problem:

Have Nexus repositories with maven artifacts and would like to configure Jenkins job to look into the repo metadata and retrieve dropdown list with available versions. 

Option 1:
Use Extensible Choice parameter from the Add Parameter menu and select System Groovy Choice Parameter. Then add the script below, click on Groovy Sandbox and theoretically this should work – well it works for Artifactory repository and only with files being available under Artifactory local  storage – for remote repositories, i.e Nexus remote repo, this did not work for me:

import groovy.json.JsonSlurper
def sortableVersion(v) {
    v.tokenize(‘.’).collect({ it.padLeft(10, ‘0’) }).join()

 def content = new URL(“//http://<artifactory_host>:<port>/artifactory/api/storage/libs-release-local/<dir>/”).text

def jsonObj = new JsonSlurper().parseText( content )
[“-“] + jsonObj.children

  .findAll { it.folder }
  .findAll { it.uri =~ /\/\d+\.\d+\.\d+/ }
  .collect { it.uri[1..-1] }
  .sort { a, b -> sortableVersion(a) <=> sortableVersion(b) }


Option 2:

Follow steps from the link below to identify correct Nexus URL and search classifiers:


This options unfortunately failed on me too due to Jenkins Script Security Plugin being enabled and current outstanding bug blocking “toURL” class to work. The groovy class does not appear on the list to be approved by the Admin in Jenkins making it almost impossible to skip. I did not give it too much though as there always has to be some other way to get what you want without too much hacking involved.

Option 3:

There is a plugin called Repository Connector

It adds Maven Repository Artifact to the drop down menu of the parameters. This one started quite a promising. However joy did not last too long. After quite some time of scratching my head it was coming back with error: artefact versions cannot be found. And rightly so, it did connect to Nexus but, the plugin itself expected pom file with metadata that contains all the versions in the repository and Nexus does not create one in this case. Instead i had metadata xml file which in no way could be read by the plugin.

Do then when the bright light shined above my head and found…

Option 4:

Maven Metadata Plugin for Jenkins CI server!!! ok – maybe it is not so exciting and very pretty but does the job for sure. It basically reads Nexus repository file and return values. It even has an option of descending the numbers and limit drop down menu to maximum versions to display. On top of that it can be nicely dressed with Artefact Info Label and few more filter options. So it looks great and works pretty well for my needs. The parameter config is quite long  which causes an overhead of scrolling down if you have many values to choose from in your job, but nevertheless it does what it says on the package so job well done!!!


Authentication failure while installing centOS image using Vagrant

Just one useful tip for those who got into trouble starting up centOS image on Vagrant.

When installing in this case “nrel/CentOS-6.5-x86_64″ image on Vagrant I have experienced timeout errors while trying to startup my boxes:

web: Warning: Authentication failure. Retrying…

This was because centOS image requires username and password while loading. Two ssh lines inside Vagrantfile helped me to solve the issue:

 Ideally after that you simply want to reload vagrant boxes using command below:

# vagrant reload 

Then check all your boxes are up:

# vboxmanage list runningvms

And that you can connect to the box using ping command.

# ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=0.425 ms
64 bytes from icmp_seq=1 ttl=64 time=0.305 ms

And you’re all set!!

Let’s start from the middle…!

It took me a while to get enough encouragement to write this first post…

For quite some time I’ve walked with the idea in my mind, that everything I do, discovering new things about the design, and not sharing it with anyone else is a great waste of really good information.  

Also, to be perfectly honest, many things I’ve been through, tend to come back to me, which makes it even more frustrating if I don’t remember how I managed to get it sorted! 

So here you go, to open this unwritten book to the public, it is a great honour to share the experience, and very often frustration, that comes along with this interesting adventure.

Let’s then dive inside the Middle and enjoy…!