Remove duplicate lines while comparing two files

I’ve been quite busy this whole day with a partially complete database dump and wanted to prepare for tomorrow with some ninja bash voodoo shizzle. I’m doing a braindump here because I know I’ll have forgotten this when I wake up tomorrow :-)

The command stated below was the first working example I’ve gotten together, please let me know if you know a neater / better solution!

The situation:

I’ve got two files. The first file contains lines which need to be deleted from the second line (if they exist there) Continue reading “Remove duplicate lines while comparing two files”

Puppet provisioning on Vagrant Error: –manifestdir

This post is a distilled version of the discussion here: https://github.com/mitchellh/vagrant/issues/3740

Problem:

When I started using the latest Puppetlabs boxes, I encountered the error “Error: Could not parse application options: invalid option: –manifestdir” when my puppet manifest was about to be executed. Continue reading “Puppet provisioning on Vagrant Error: –manifestdir”

Create a simple HTTPS server with OPENSSL S_SERVER

This post will mostly serve as a reference for future posts, the goal is to create the simplest HTTPS webserver possible, which will serve to test certificates, authentication via private keys and in the end; configure SSL offloading to an Apache HTTPD, which will act as a proxy between your client and the secure endpoint.

GOAL: At the end of this article, you will have a running secure web server which you can access via your web browser and/or via an SSL client. Continue reading “Create a simple HTTPS server with OPENSSL S_SERVER”

Remove host from SSH KnownHosts file without seeing the hostname

This post is mostly a bookmark for myself. I’ve been using search engines way too often to find this command..

The command to remove a host from a knownhosts file without seeing the actual hostname in the knownhosts file is the following:

ssh-keygen -R HOSTNAME

Continue reading “Remove host from SSH KnownHosts file without seeing the hostname”

Comparing sed stream output in linux

Sed is very very powerful, which is a good thing to be aware of.
I was looking to compare the output of a sed command to the original file before I wanted to execute the sed command directly on the file and came across this handy trick.

It works by using temporary named pipes inside the diff command.

Contents of file:

One
Two
Three
Four
Five

If I just want to remove the line which begins with “Four”, I can check my sed command like this:

joris@beanie ~
$ diff <(sed '/Four/d' numbers.txt) numbers.txt
3a4
> Four

Awesome possum, now I know my sed command won’t destroy anything.

Easily switch between java versions using alternatives in Linux

This approach works in several distros, I’ve been using it in Ubuntu for a while and just used it in Fedora as well.

As a developer, you might need to switch between java versions often, this approach will come in handy then.

We will be using the command “alternatives”, in this case to check the configuration of your Java installation. The default is most often OpenJDK, while you might need Oracle Java.

Run “alternatives –display java” to see which versions you can currently choose from:

[joris@today ~]$ alternatives --display java
java - status is manual.
 link currently points to /usr/java/latest/bin/java
Current `best' version is /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.31-3.b13.fc21.x86_64/jre/bin/java.
[joris@today ~]$ 

There’s no Oracle Java yet, make sure you’ve installed Oracle Java. If you haven’t, you can check this blog post: Install Oracle Java in Fedora, Red Hat or CentOS using Yum and RPM

When Oracle Java is installed, you can add it to your alternatives: “sudo alternatives –install /usr/bin/java java /usr/java/latest/bin/java 20000”

Please note: I used “latest” in the command above, another options is to specifically set the version you want. This way you can install several JDK’s and switch as shown below.

When that is finished, you can select your current flavour of Java:

[joris@today ~]$ sudo alternatives --config java

There are 2 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
*  1           /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.31-3.b13.fc21.x86_64/jre/bin/java
 + 2           /usr/java/latest/bin/java

Enter to keep the current selection[+], or type selection number: 

Choose the option you want to switch between Java versions.

As mentioned by enkouyami, please check if you need to use update-java-alternatives instead of alternatives. The use of alternatives was valid when I wrote the post, but might not be working anymore!

Bash: Shortcuts to your favorite directories with CDPATH

This is a fairly handy trick.

I’m always logging in to different servers, which all have a different location for their logs. With this trick you can login and just type “cd logs” from anywhere.

First we will add our directory to the CDPATH variable:

joris@badattitude /data/share/my_domain
$ echo $CDPATH

joris@badattitude /data/share/my_domain
$ export CDPATH=/data/share/my_domain

joris@badattitude /data/share/my_domain
$ echo $CDPATH
/data/share/my_domain

Then we’ll check if it is working:

joris@badattitude ~
$ cd logs
/data/share/my_domain/logs

joris@badattitude /data/share/my_domain/logs
$

Awesome, we’ve arrived in our logs directory straight from our homedir!

SSH Remote Execute command, multiple command and with interaction

This article shows how to execute remote commands via ssh, but you’ll send the commands from your own shell.

ssh my_server 'ls -l /home/my_home_dir'

This will result in this output:

$ ssh ae2 'ls -lha ~'
total 36K
drwxr-xr-x  2 joris joris 4.0K Jan 23 11:42 .
drwxr-xr-x. 5 root  root  4.0K Jan 23 11:41 ..
-rw-r--r--  1 joris joris   54 Jan 23 11:41 .bash_logout
-rw-r--r--  1 joris joris  507 Jan 23 11:41 .bash_profile
-rw-r--r--  1 joris joris  213 Jan 23 11:41 .bashrc
-rw-------  1 joris joris   51 Jan 23 11:42 .history
-rw-r--r--  1 joris joris  171 Jan 23 11:41 .kshrc
-rw-r--r--  1 joris joris  375 Jan 23 11:41 .profile
-rw-r--r--  1 joris joris  153 Jan 23 11:41 .vimrc

What’s even better, is that you can run multiple commands separated with a semi colon, like this:

ssh my_server 'ls -l /home/my_home_dir;whoami'

And the best trick is this one, user input with an interactive command, sending input and output back and forth!

ssh -t my_server 'vi ~/.bash_profile'

Find Java JRE location on Ubuntu Linux

Everybody knows that the java executable is located in /usr/bin/java , but what if you need the JDK / JRE location itself?

Just using “whereis” will not get you there, that will point you to the /usr/bin/java point.

joris@howlingmad: ~_011

 

So, let’s find out a but more about /usr/bin/java:

ls -l /usr/bin |grep java

joris@howlingmad: ~_012

 

Awesome, this will lead us somewhere, it’s a symlink to /etc/alternatives/java

So let’s do the same there:

ls -l /etc/alternatives/ |grep java

And we’ve hit the jackpot, among the lines here, there’s a bunch of lines pointing us to the JRE location:

joris@howlingmad: ~_013

 

As you can see in the screenshit, our java executable within the JRE location is:

/usr/lib/jvm/java-7-oracle/jre/bin/java

Which means the JRE location is:

/usr/lib/jvm/java-7-oracle/jre

 

 

 

SSH: Different settings (keys!) for different hosts

There’s a simple way to create aliases in a SSH config file. This way you can connect way easier to different hosts, combined with this blog post to use keys to log in.

Where you used to use this connect string:

ssh pi@192.168.0.5
pi@192.168.0.5's password: <<enter boresome password>>

You can now just enter

ssh pi

Where “pi” is the alias that you’ll be using!

The only thing you’ll need to do is create the following file:

[[ ~/.ssh/config ]]

Host pi
  HostName 192.168.0.5
  User pi
  << (!) Only add below line if you're using keys to log in >>
  IdentityFile ~/.ssh/keys/raspberry_key

Aint that awesome!?

Ps. If you’re still being asked to enter your password, check your keys and please look at this blog post I wrote.