How to Use Java Expressions in JBoss EAP System Properties

On my current client, they are using JBoss EAP 6.1.1 regardless of my incessant  pressure for them to upgrade to the latest and greatest. Since they are using an older version of EAP they can not leverage Java Expressions in EAP system properties. To some this may not cause a problem, however when your application is trying abstract, say the name of attributes that are vaulted across different environments and/or leverage the system property in more than place within the app servers configuration; there are other use cases as well.

I applied a patch that provides this functionality, but there was no clear direction on how to actually add Java expressions to the configuration file and after much chagrin I was able to decipher how to do it and am going to share it for others.

How to use system properties with vaulted attributes

Create keystore

keytool -genkeypair -v -alias alias -keyalg RSA  -keysize 2048 -dname "cn=blah,ou=device, ou=service" -keypass password -keystore myserver.keystore -storepass password

Add servers keystore to vault as an attribute

 $JBOSS_HOME/bin/vault.sh --keystore vault.keystore --keystore-password 'vault-password' --alias vault --enc-dir  --salt 11111111 --iteration 10 -a keystore_name -x 'myserver.keystore'

********************************************
Vault Block:vb
Attribute Name:keystore_name
Configuration should be done as follows:
VAULT::vb::keystore_name::1
********************************************
Vault Configuration in AS7 config file:
********************************************
...
</extensions>
<vault>
  <vault-option name="KEYSTORE_URL" value="vault.keystore"/>
  <vault-option name="KEYSTORE_PASSWORD" value="MASK-1dmPKRStsI..BzvEbFkZi"/>
  <vault-option name="KEYSTORE_ALIAS" value="vault"/>
  <vault-option name="SALT" value="11111111"/>
  <vault-option name="ITERATION_COUNT" value="10"/>
  <vault-option name="ENC_FILE_DIR" value=""/>
</vault><management> ...
********************************************

Add keystore password to vault

$JBOSS_HOME/bin/vault.sh --keystore vault.keystore --keystore-password 'vault-password' --alias vault --enc-dir  --salt 11111111  --iteration 10 -a keypass -x 'password'

********************************************
Vault Block:vb
Attribute Name:keypass
Configuration should be done as follows:
VAULT::vb::keypass::1
********************************************
Vault Configuration in AS7 config file:
********************************************
...
</extensions>
<vault>
  <vault-option name="KEYSTORE_URL" value="vault.keystore"/>
  <vault-option name="KEYSTORE_PASSWORD" value="MASK-1dmPKRStsI..BzvEbFkZi"/>
  <vault-option name="KEYSTORE_ALIAS" value="vault"/>
  <vault-option name="SALT" value="11111111"/>
  <vault-option name="ITERATION_COUNT" value="10"/>
  <vault-option name="ENC_FILE_DIR" value=""/>
</vault><management> ...
********************************************

Add user

$JBOSS_HOME/bin/add-user.sh --silent admin admin.2015

Start server

$JBOSS_HOME/bin/standalone.sh

Add system property <kstore-name>

$JBOSS_HOME/bin/jboss-cli.sh --user=admin --password=admin.2015 -c controller=${HOSTNAME}:9999 --command='/system-property=kstore-name:add(value=$\\{VAULT::vb::keystore_name::1\})'

Add ssl

add_keystore_cmd='/subsystem=web/connector=https/ssl=configuration:write-attribute(name=certificate-key-file,value="${kstore-name}")'
$JBOSS_HOME/bin/jboss-cli.sh --user=admin --password=admin.2015 -c controller=${HOSTNAME}:9999 --command="${add_keystore_cmd}"

tail logs

tail -f $JBOSS_HOME/standalone/log/server.log &

reload server

$JBOSS_HOME/bin/jboss-cli.sh --user=admin --password=admin.2015 -c controller=${HOSTNAME}:9999 --command="reload"
Advertisements

How to secure bash command line history

Lately I have become of aware of an unsafe practice when maintaining and implementing Jboss EAP servers (or any servers really). The idea is that whatever commands that are executed as a particular user are persisted to the ~/.bash_history file. Of course history can be disabled for particular users, but not all system administrators take this into consideration and hence the concern.

Why is this dangerous

If there is a security breach into a system, say with sudo privileges, the attacker will not only have access into the current system, but also bash_history files. The bash_history wrt JBoss EAP, is that the server uses several cli tooling to create server certificates, management users, vaults, and vaulted attributes and if care isn’t taken can lead to information leaks.

Example

As a user I create a keystore, vault and add secure vault attributes

Login as good guy

[root@mybox ~]# sudo su - JbossAdm

Create keystore

[JbossAdm@mybox jboss-eap-6.3]$ keytool -genseckey -v -alias vaultAlias -keyalg AES -keysize 128 -storetype jceks -dname "CN=something" -keypass vaultKeyPass -keystore vault.jks -storepass vaultKeyPass

Create vault

[JbossAdm@mybox jboss-eap-6.3]$ ./bin/vault.sh --keystore vault.jks --keystore-password vaultKeyPass --alias vaultAlias --salt 12457898 --iteration 15 -c

Create vault attribute

[JbossAdm@mybox jboss-eap-6.3]$ ./bin/vault.sh --keystore vault.jks --keystore-password vaultKeyPass --alias vaultAlias --salt 12457898 --iteration 15 --attribute certKeystorePass --sec-attr hostKeyPass

Add management user

[JbossAdm@mybox jboss-eap-6.3]$ ./bin/add-user.sh --silent -u admin -p admin.2015

Nefarious user Login

It is fair to say, having root access alone does not guarantee visability into the passwords of vault, keystore or attributes. Look how easy it is to get passwords for JBoss EAP and server certificate store. With this information they can do untold damange.

Interogate bash history for keywords

[root@mybox ~]# grep '.*vault.*\|.*keytool.*\|.*add-user.*' /home/JbossAdm/.bash_history
./bin/add-user.sh --silent -u admin -p admin.2015
keytool -genseckey -v -alias vaultAlias -keyalg AES -keysize 128 -storetype jceks -dname "CN=something" -keypass vaultKeyPass -keystore vault.jks -storepass vaultKeyPass
./bin/vault.sh --keystore vault.jks --keystore-password vaultKeyPass --alias vaultAlias --salt 12457898 --iteration 15 -c
./bin/vault.sh --keystore vault.jks --keystore-password vaultKeyPass --alias vaultAlias --salt 12457898 --iteration 15 --attribute certKeystorePass --sec-attr hostKeyPass

How to prevent (at least slow down nefarious acts)

Since this scenario is in a data center that has been around for some time and who knows what has been run prior to this box, first state of affairs to clear history.

Remove only suspect values

This looks complicated, but it cats the history file, reverses order then greps for key values and the cuts the number associated with, then loops through and deletes each of the records found.

[JbossAdm@mybox ~]$ histnum=$(history | tac | grep '.*vault.*\|.*keytool.*\|.*add-user.*|.*jboss-cli.*' | sed 's/^[ ]*//;s/[ ].*//;p')
[JbossAdm@mybox ~]$ for del in $histnum; do  history -d $del; done
[JbossAdm@mybox jboss-eap-6.3]$ history -w
Note
tac is needed other wise the ordering is inaccurate after the first removal and will extend past the total number of elements (in some cases), but with reverse order always starts with the last value and works to smallest ensuring they are always there.

Alternately, we could clear current history

[JbossAdm@mybox jboss-eap-6.3]$ history -c
[JbossAdm@mybox jboss-eap-6.3]$ history -w

Set HISTIGNORE

Now, we can do this each time we have a server configured, but the ideal solution is to have it built as part of the VM profile so it is handled automatically. For more information on HISTIGNORE visit reference [1].

[JbossAdm@mybox jboss-eap-6.3]$ HISTIGNORE='keytool[ ]*':'*vault*':'*add-user*':'*jboss-cli*'
[JbossAdm@mybox jboss-eap-6.3]$ export HISTIGNORE

Execute commands again

I am supressing the output, because it really isn’t important to this example.

[JBossAdm@mybox jboss-eap-6.3]$ keytool -genseckey -v -alias vaultAlias -keyalg AES -keysize 128 -storetype jceks -dname "CN=something" -keypass vaultKeyPass -keystore vault.jks -storepass vaultKeyPass
[Storing vault.jks]
[JbossAdm@mybox jboss-eap-6.3]$ ./bin/vault.sh --keystore vault.jks --keystore-password vaultKeyPass --alias vaultAlias --salt 12457898 --iteration 15 -c
[JbossAdm@mybox jboss-eap-6.3]$ ./bin/vault.sh --keystore vault.jks --keystore-password vaultKeyPass --alias vaultAlias --salt 12457898 --iteration 15 --attribute certKeystorePass --sec-attr hostKeyPass
[JbossAdm@mybox jboss-eap-6.3]$ ./bin/add-user.sh --silent -u admin -p admin.2015

list history

As we expect we see no values with vault, keytool or add-user

[root@mybox ~]# grep '.*vault.*\|.*keytool.*\|.*add-user.*' /home/JbossAdm/.bash_history

Conclusion

For the experienced and dedicated it is merly a road bump, but that is okay. This is merly meant to slow down and deter any nefarious activity.