python + ssh (paramiko) + N servers + M commands

Recently I was asked by a client to basically run a block of commands on 100+ servers. All the servers had the same commands to be executed as well as my user account was the same for all the boxes, which is convenient.

Anyways, when they asked me to do this they had no automated scripts to do this ( we’re modernizing their environment 🙂 ) and I’m thinking to myself, ‘there is no way I’m going to manually log into these boxes to run these commands’. Then I briefly contemplated whether or not I wanted to write a bash script to do the job… And I was like no thanks. Basically it was for the same reasons why I avoid bash at all costs (clunky, cryptic and touchy), no offense my bash brethren.

Python, from my perspective is amazing. It is easy to use, easy to understand, Python has a huge library (modules), user community is vast, and to do complicated sys admin or development you don’t need be a wizard. I have a slew of colleagues who swear by ruby and I have used it briefly, but it just doesn’t have the same feel and intuitiveness that Python does. WTS, I bet once I learned the syntax better I’d probably like it equally was well, but eh, for now Python and it does everything I want/need, quickly!

Backkkk to my post about python + ssh (paramiko), okay whilst reading Python for Unix and Linux System Administration by Noah Gift and Jeremy Jones (great book, highly recommend) on SSH tools I came across Paramiko. And wow, what a great simple way to extend ssh tooling to Python.

Using Paramiko I was able to set up a script that churned through all 100+ servers with a list of commands and log the results in a matter of minutes. Of course this doesn’t include testing, because I didn’t want to test on live systems at least not if I wanted to keep my contract 🙂 .

Check out my code on github, it is extremly straightforward (assuming you are familiar with the basic tenants of SSH and remote administration).

https://github.com/jmarley/python-ssh-remote-cmd-exec

Happy coding!

installing Openshift v2: devil’s in the details

This past several weeks I’ve been venturing into Red Hat’s Cloud PAAS technology, Openshift. I attempted to install the community version (origin) and the enterprise version. I had mixed luck. With the community version the documentation was incredible, I mean a monkey could do the installation based off of their instructions, really is top notch. Great Job guys! I ran into some packages not being quite stable so i decided then try the enterprise distribution.

So not being an expert (yet!) in Installing/Configuring openshift I needed to rely on the instructions heavily, don’t judge 🙂 . I had no real issues to speak of doing the installation until I installed the broker console. Then I was getting this friendly error:

[root@broker ~]# service openshift-broker start
Starting openshift-broker: Unable to open /etc/scl/prefixes/v8314!
chown: cannot access `Gemfile.lock': No such file or directory
Bundle failed
Run 'bundle install --local' in /var/www/openshift/broker/ to see the problem.

Being a newbie to Red Hats Software Collections I was not sure what the error was and/or why I was getting it. So I cracked open the code base and noticed that the version of ruby I was using was calling this cryptically named command v8134 and without fail couldn’t figure out what it was. I asked a few of my colleagues, but to no avail came up short.

soooo, I checked to see if anybody else had come across this issue before and when the whole error is entered no dice, so I started taking bits off and only leaving the core part of the error and like magic I found the bug had been reported by someone else (https://bugzilla.redhat.com/show_bug.cgi?id=1108762). Turns out the RPM file accidentally left off that dependency. And manually installing the v8134 with yum install fixed my issue. All I can say is awesome! Because I could finally move on.

I was able to set up my host no problem and now I’m ready for consumers….jk

lesson learned always google for bugs, trim out error and/or go to vendors site and search their known bugs.

How to deploy exploded archives JBoss EAP 6

This is a how to deploy an archive in an exploded format in either standalone or domain mode, albeit I am only going to show standalone.

Deploy application
This is the process to deploy an exploded archive

Build Archive

I will not go through the mechanics of creating an archive, but for my quick how to I used the helloworld war example from JBoss.org

  1. Archive Name: jboss-helloworld.war
  2. Create exploded archive
$ cd jboss-helloworld.war/
  1. Extract archive into directory
$ jar -xvf ~/projects/community/jboss-as-quickstart/helloworld/target/jboss-helloworld.war
  1. Copy base configuration Good practice for testing
$ cd /../jboss-eap-6.1
$ cp ./standalone/configuration/standalone.xml ./standalone/configuration/exploded_test.xml
  1. Start EAP
[jboss-eap-6.1]$ ./bin/standalone.sh -c exploded_test.xml
Deploy Archive
[jboss-eap-6.1]$ cp -r /workspace/exploded_archives/jboss-helloworld.war/ ./standalone/deployments/
  1. Review output The application should have deployed correctly and to verify review output.
11:44:47,471 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 2) JBAS015003: Found jboss-helloworld.war in deployment directory. To trigger deployment create a file called jboss-helloworld.war.dodeploy
Take notice how nothing was actually deployed when an exploded archive is copied into the deployments folder
Create dodeploy file

Basically we need to tell the server to explode a particular archive and the way we do this is creating a dodeploy for a particular archive, otherwise EAP will keep a reference to the archive as <archive-name>.<archive-extension> under the deployments folder.

$ touch ./standalone/deployments/jboss-helloworld.war.dodeploy
  1. Review output
11:47:27,525 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-3) JBAS015876: Starting deployment of "jboss-helloworld.war" (runtime-name: "jboss-helloworld.war")
11:47:27,560 INFO  [org.jboss.weld.deployer] (MSC service thread 1-1) JBAS016002: Processing weld deployment jboss-helloworld.war
11:47:27,571 INFO  [org.jboss.weld.deployer] (MSC service thread 1-4) JBAS016005: Starting Services for CDI deployment: jboss-helloworld.war
11:47:27,582 INFO  [org.jboss.weld.deployer] (MSC service thread 1-2) JBAS016008: Starting weld service for deployment jboss-helloworld.war
11:47:27,635 INFO  [org.jboss.web] (ServerService Thread Pool -- 75) JBAS018210: Register web context: /jboss-helloworld
11:47:27,719 INFO  [org.jboss.as.server] (DeploymentScanner-threads - 1) JBAS018559: Deployed "jboss-helloworld.war" (runtime-name : "jboss-helloworld.war")
Verify exploded archive
[jboss-eap-6.1]$ tree standalone/deployments/
standalone/deployments/
├── jboss-helloworld.war
│   ├── index.html
│   ├── META-INF
│   │   ├── MANIFEST.MF
│   │   └── maven
│   │       └── org.jboss.quickstarts.eap
│   │           └── jboss-helloworld
│   │               ├── pom.properties
│   │               └── pom.xml
│   └── WEB-INF
│       ├── beans.xml
│       └── classes
│           └── org
│               └── jboss
│                   └── as
│                       └── quickstarts
│                           └── helloworld
│                               ├── HelloService.class
│                               └── HelloWorldServlet.class
├── jboss-helloworld.war.deployed
└── README.txt

12 directories, 9 files
if the archive is updated the .dodeploy also needs to have an updated time stamp. The easiest way is to simply retouch the file *touch .dodeploy*

Learning Python 3 Quickly via Head First Python

So at my office there is a lot of buzz around Ruby and Python. Both have their pros/cons, but I ended up choosing python to tackle first. I wouldn’t classify myself as a developer, but I am familiar with writing code and the concepts behind best practices. A vacillated between Python 2 or 3 and did a lot of reading and the consensus was if “you” know python 3 it is easy to go backwards, but learning concepts from 2 and going to three you have to unlearn and is more difficult.. I can attest this is the case.

Anyways, I choose Python 3 and not being at all familiar with Python other than it is a scripting language I wanted to test the waters with an easy technical guide to get some foundational concepts and just start using it (that is the best way..right :] ).

Well I have used had headfirst books before in the past and hadn’t really found them overlly useful (albeit it may have been because I was already familiar with those technologies), but a colleague suggested Head First Python, by Paul Barry and I gave it a look and seemed to have everything I was looking for and some.

I can’t say enough positives about it. Within minutes of reading I was already coding. The explanations and depth fit perfectly with what we were doing. Like I mentioned above, I am familiar with coding so I didn’t need to spend a lot of time on learning the fundamentals, but Paul does a good job of pointing out areas of where to learn more and keeps moving so you don’t get bogged down in the minutia of computer science (which is cool to, but not at a first pass). The examples all worked and I learned a lot!

I would recommend to follow what Paul says as far as following along all the way throughout the book; I was tempted several times to skip ahead, but at the end of the day I was glad I took my time and went through the examples even though I already knew how to do stuff.