Couchbase and Teiid Bang!

Over the last few months I’ve been creating communities within our consulting teams. There are a lot of consultants that are very interested in JBoss Data Virtualization (a.k.a. Teiid in the JBoss Community)  and over the last few months I’ve been putting together a team of consultants with varying interests to get them familiar with this project/product.

So, for those of you who may not be familiar with Teiid(JDV), it has many powerful features and one of the most powerful is the ability to extend the product. Teiid comes with many translators that can bridge ANSI sql queries to NoSQL, traditional SQL, flat files and more types of data sources, however, some translators don’t exist for all data sources. In comes the community: massive exaggerated CHEERING. queue sound byte.  My team and I have been focused on extending Teiid to be able to make ANSI SQL translations between Couchbase, which itself is a layer on top of Couchdb. There is a thread as to why I ended up choosing Couch.

Also we choose Couchbase primarily from its ease of use. Couchbase comes with NQL, which is a very SQLish like api, so doing the translations from SQL to NQL is much simpler rather than working with a who new api with lots of switches and exceptions and most important having to learn it all before we start. My main goal was/is to build a foundation where we can branch out into more complicated data sources.

Here we are. It has taken a bit of time to get the project stood up in a repeatable fashion, such that I can add new team members without much time for set up. We have docker images set up for JDV, and Couchbase so it is a matter of just issuing a few docker commands to get the servers up. Currently, we have the resource adapter working and will soon be actually doing the translations with NQL. To get started with this project (please help 🙂 ) you can pull from the main teiid branch.


$ mkdir teiid-couchbase

$ cd teiid-couchbase

$ git init

$ git remote add upstream github.com:teiid/teiid.git

$ git checkout -b couchbase upstream/8.12.x

Now you have the source for couchbase translator, now lets set up your environment


$ git clone https://github.com/jmarley/dv-couchbase

You should be able to follow the instructions on standing it up. If you see any issues let me know and do a pull request. That’s it! I’ll update when I have some news about this project.

 

Advertisements

How to Use Java Expressions in JBoss EAP System Properties

On my current client, they are using JBoss EAP 6.1.1 regardless of my incessant  pressure for them to upgrade to the latest and greatest. Since they are using an older version of EAP they can not leverage Java Expressions in EAP system properties. To some this may not cause a problem, however when your application is trying abstract, say the name of attributes that are vaulted across different environments and/or leverage the system property in more than place within the app servers configuration; there are other use cases as well.

I applied a patch that provides this functionality, but there was no clear direction on how to actually add Java expressions to the configuration file and after much chagrin I was able to decipher how to do it and am going to share it for others.

How to use system properties with vaulted attributes

Create keystore

keytool -genkeypair -v -alias alias -keyalg RSA  -keysize 2048 -dname "cn=blah,ou=device, ou=service" -keypass password -keystore myserver.keystore -storepass password

Add servers keystore to vault as an attribute

 $JBOSS_HOME/bin/vault.sh --keystore vault.keystore --keystore-password 'vault-password' --alias vault --enc-dir  --salt 11111111 --iteration 10 -a keystore_name -x 'myserver.keystore'

********************************************
Vault Block:vb
Attribute Name:keystore_name
Configuration should be done as follows:
VAULT::vb::keystore_name::1
********************************************
Vault Configuration in AS7 config file:
********************************************
...
</extensions>
<vault>
  <vault-option name="KEYSTORE_URL" value="vault.keystore"/>
  <vault-option name="KEYSTORE_PASSWORD" value="MASK-1dmPKRStsI..BzvEbFkZi"/>
  <vault-option name="KEYSTORE_ALIAS" value="vault"/>
  <vault-option name="SALT" value="11111111"/>
  <vault-option name="ITERATION_COUNT" value="10"/>
  <vault-option name="ENC_FILE_DIR" value=""/>
</vault><management> ...
********************************************

Add keystore password to vault

$JBOSS_HOME/bin/vault.sh --keystore vault.keystore --keystore-password 'vault-password' --alias vault --enc-dir  --salt 11111111  --iteration 10 -a keypass -x 'password'

********************************************
Vault Block:vb
Attribute Name:keypass
Configuration should be done as follows:
VAULT::vb::keypass::1
********************************************
Vault Configuration in AS7 config file:
********************************************
...
</extensions>
<vault>
  <vault-option name="KEYSTORE_URL" value="vault.keystore"/>
  <vault-option name="KEYSTORE_PASSWORD" value="MASK-1dmPKRStsI..BzvEbFkZi"/>
  <vault-option name="KEYSTORE_ALIAS" value="vault"/>
  <vault-option name="SALT" value="11111111"/>
  <vault-option name="ITERATION_COUNT" value="10"/>
  <vault-option name="ENC_FILE_DIR" value=""/>
</vault><management> ...
********************************************

Add user

$JBOSS_HOME/bin/add-user.sh --silent admin admin.2015

Start server

$JBOSS_HOME/bin/standalone.sh

Add system property <kstore-name>

$JBOSS_HOME/bin/jboss-cli.sh --user=admin --password=admin.2015 -c controller=${HOSTNAME}:9999 --command='/system-property=kstore-name:add(value=$\\{VAULT::vb::keystore_name::1\})'

Add ssl

add_keystore_cmd='/subsystem=web/connector=https/ssl=configuration:write-attribute(name=certificate-key-file,value="${kstore-name}")'
$JBOSS_HOME/bin/jboss-cli.sh --user=admin --password=admin.2015 -c controller=${HOSTNAME}:9999 --command="${add_keystore_cmd}"

tail logs

tail -f $JBOSS_HOME/standalone/log/server.log &

reload server

$JBOSS_HOME/bin/jboss-cli.sh --user=admin --password=admin.2015 -c controller=${HOSTNAME}:9999 --command="reload"

How Snapshoting works with EAP 6

This simple little guide shows more or less how a snapshot works with EAP 6. I’ll show what happens to a deployable when a snapshot is taken between another version of the app without a name change and the newer version is deployed on the same server.

I am using a standard configuration file from JBoss EAP 6.1.1, though the same will hold true for any configuration file.

Use Case

  1. Deploy war, take snapshot, deploy updated archive with different name and same runtime name, disable initial war, reload,
Deploy war archive
$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy --name=snap1 snapshot-test.war --runtime-name=snapshot-test.war"
verify deployment
$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy -l"
NAME  RUNTIME-NAME      ENABLED STATUS
snap1 snapshot-test.war false   STOPPED
Take snapshot
$ JBOSS_HOME/bin/jboss-cli.sh -c --command=":take-snapshot"
{
    "outcome" => "success",
    "result" => "/opt/app/jboss/jboss-eap-6.1/standalone/configuration/standalone_xml_history/snapshot/20141104-12283814720141104-113544608snapshot-uc-1.xml"
}
Disable War

The archive needs to be disabled or else the container will simply use the first instance of the war, since both of the deployables will have the same name.

$ JBOSS_HOME/bin/jboss-cli.sh -c --command="/deployment=snap1:undeploy()"
Deploy second archive
$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy --name=snap2 tweaked-war/snapshot-test.war --runtime-name=snapshot-test.war"
verify deployment
$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy -l"
NAME  RUNTIME-NAME      ENABLED STATUS
snap1 snapshot-test.war false   STOPPED
snap2 snapshot-test.war true    no metrics available
You will may have to reload the server for the container to recongnize the newer reference to the archive
Take snapshot
$ JBOSS_HOME/bin/jboss-cli.sh -c --command=":take-snapshot"
{
    "outcome" => "success",
    "result" => "/opt/app/jboss/jboss-eap-6.1/standalone/configuration/standalone_xml_history/snapshot/20141104-12315947120141104-113544608snapshot-uc-1.xml"
}
Stop current instance of EAP
$ JBOSS_HOME/bin/jboss-cli.sh -c --command="shutdown"
Start EAP with first snapshot
$ JBOSS_HOME/bin/standalone.sh -c standalone_xml_history/snapshot/20141104-12283814720141104-113544608snapshot-uc-1.xml -b 0.0.0.0 &
verify deployment
$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy -l"
NAME                 RUNTIME-NAME         ENABLED STATUS
snap1                snapshot-test.war    true    OK
Stop current instance of EAP
$ JBOSS_HOME/bin/jboss-cli.sh -c --command="shutdown"
Start EAP with second snapshot
$ JBOSS_HOME/bin/standalone.sh -c standalone_xml_history/snapshot/20141104-12315947120141104-113544608snapshot-uc-1.xml -b 0.0.0.0 &
verify deployment
$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy -l"
NAME                 RUNTIME-NAME         ENABLED STATUS
snap1                snapshot-test.war    false   STOPPED
snap2                snapshot-test.war    true    no metrics available

disabling deployed app on Jboss EAP 6

Generally, I don’t write a blog post about something so simple as a 1 off, but seeing as others before me haven’t figured this out and I had only stumbled upon how it works after the fact of some unfruitful researching.

If you have an application deployed on EAP 6 and want to disable it from the CLI tool, look no further. It is very easy and counter intuitive, but it works and you can verify by looking at the management console, which is what I used to disable before I found out how to do it on the command line.

Deploy application

$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy snapshot-test.war"

Verify


$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy -l"

NAME              RUNTIME-NAME      ENABLED STATUS
snapshot-test.war snapshot-test.war true    OK

Disable application


$ JBOSS_HOME/bin/jboss-cli.sh -c --command="/deployment=snapshot-test.war:undeploy()"

Verify


$ JBOSS_HOME/bin/jboss-cli.sh -c --command="deploy -l"

NAME              RUNTIME-NAME      ENABLED STATUS
snapshot-test.war snapshot-test.war false   STOPPED

Conclusion

Why confusion, well the name for one. I guess it makes sense wrt the container, however, if you run


$ JBOSS_HOME/bin/jboss-cli.sh -c --command="undeploy snapshot-test.war"

The war content will be completed removed from the server, vice running undeploy from the /deployment=<name>:undeploy() operation, where it will just be disabled and not removed from the server.