Hi. I’m Rudy Lee.

Here are some thoughts of mine.

Git

Reopen Last Commit in Git

Run the command below if you want to reopen the last commit in your git

1
git reset --soft HEAD~

This is useful if you miss something in your last commit. Instead of creating new commit and squashing it, you can open last commit and fix it there.


AWS

Copying Files Between Two S3 Buckets

Today, I have to copy files from one S3 bucket to another S3 bucket which sitting in separate AWS account.

Initially, I was thinking to use S3 clients ( Tranmit or Cyberduck ) to download the files first and manually upload the files again to the other S3 Bucket. However, this approach will consume a lot of bandwidth and really slow if you have a lot of files in your S3 bucket.

After a bit of research, I found that you can easily copy files between two S3 buckets. You can use either s4cmd or AWS CLI.

s3cmd or s4cmd

Run this command to install s4cmd

1
pip install s4cmd

After you finished setting up the AWS credentials, you can start the copying process.

1
2
3
4
5
# format
s4cmd cp s3://<source-bucket> s3://<target-bucket>/ --recursive

#example
s4cmd cp s3://rudylee-images s3://rudylee-new-images/ --recursive

AWS CLI

Install the cli through pip

1
pip install awscli

And configure it

1
awscli configure

The usage is quite similar to s4cmd, see below:

1
aws s3 cp s3://<source-bucket> s3://<destination-bucket>

I prefer using AWS CLI because it has more options and official support. AWS CLI has a built it support can specify the ACL and permission of the objects.

1
2
# The command below will allow the target bucket owner to have full access to the object
aws s3 cp s3://<source-bucket> s3://<target-bucket> --acl "bucket-owner-full-control" --recursive

Since my target bucket is sitting in separate AWS account, I have to set another permission to allow everyone to upload and delete files into my target bucket.

If you want to follow this approach, make sure to delete that permission after you finished with the copying.

The other option is to set the S3 bucket policy manually, see this link: http://serverfault.com/questions/556077/what-is-causing-access-denied-when-using-the-aws-cli-to-download-from-amazon-s3


Create SSH Tunnel To Backup PostgreSQL Database

Today, I was trying to create a backup of production database. The problem is we have two different version of PostgreSQL running on production and these databases can only be accessed from front end(FE) server. We have older version of PostgreSQL client installed in all FE server which means I can’t use it to run pg_dump.

SSH Tunnel to the rescue

One solution to this problem is to create a SSH tunnel. Since I have the latest version of PostgreSQL client installed in my machine, I can run pg_dump locally which will connect to the database through SSH tunnel.

Here is the command I used to create SSH tunnel:

1
2
# Format: ssh -L <local-port>:<db-hostname>:<db-port> <fe-username>@<fe-hostname>
ssh -L 9000:database.com:5432 ubuntu@production.servers.com

After this you can check whether the SSH tunnel is succesfully created by running this and look for port 9000

1
netstat -na | grep LISTEN

If you confirm that the SSH tunnel is working, you can run psql to connect or pg_dump to backup your database

1
psql -h localhost -p 9000

Rails Engines and Vim

At the moment, I am working on a Ruby on Rails projects using Rails Engines ( you can read more about Rails Engines here: http://guides.rubyonrails.org/engines.html ). In this post, I’ll share my tips and trick on how to configure your vim to work with Rails Engines.

NERDTree Bookmark

I am using NERDTree Bookmark to quickly jump between different engines. If you are using NERDTree, you can create a bookmark by putting your cursor on one of the Rails Engines directory and use the command below:

1
:Bookmark <engine name>

After you created the bookmark, you can see the bookmarks list by pressing B inside NERDTree window. See the screenshot below:

I also added these two options to my vimrc file.

1
2
3
" Automatically show bookmarks list when you open NERDTree
let NERDTreeShowBookmarks=1
let NERDTreeChDirMode=2

NERDTreeChDirMode changes the current working directory of your vim to your bookmark directory. This will also enable my favourite rails.vim feature which is open alternate file.

CtrlP Working Path Mode

It is normal for Rails Engines to share similar directory structure and filenames. However, this creates problem when you want to search a file using CtrlP plugin. Combined with NERDTreeChDirMode, you can tell CtrlP to search only in the current working directory.

Add this option to your vimrc file to enable this feature:

1
let g:ctrlp_working_path_mode = 'a'

That’s it for now, I’ll update this post if I find a better workflow or configuration. If you are interested, you can check my full vimrc file here: https://github.com/rudylee/dotfiles/blob/master/vimrc


Symbolic Links With Vagrant Windows

There are few known limitations when using Vagrant on Windows machine. One of these limitations is the lack of symbolic links support on synced folder.

Symbolic links are used heavily by NPM to create shortcut for the libraries. I posted more details about this here: http://blog.rudylee.com/2013/10/24/fix-npm-symlink-problem-in-vagrant/

Most of the time, you can get away with ‘npm —no-bin-link’ solution. However, you need more robust solution if you are using complex tools such as Grunt or Yeoman.

In this post, I’ll show you the proper way to add symbolic links support to your Vagrant machine.

First, you need to add this code snippet inside your Vagrantfile

1
2
3
config.vm.provider "virtualbox" do |v|
    v.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"]
end

VirtualBox disables symbolic links for security reasons. In order to pass this restriction, you need to boot up the Vagrant machine in Administrator mode.

You can do this by simply right clicking on your Command Prompt or Git Bash icon and click ‘Run as Administrator’. See the picture below if you can’t find it.

After that, boot up the Vagrant machine normally with ‘vagrant up’ command. Wait until the machine boots up, SSH to the machine and try to create symbolic link in the synced folder.

File path 255 character limit

Another annoying problem you might encounter is file path character limit. This happens quite often if you are using a node module with long name. You can easily solve this by following these steps:

Create ‘node_modules’ folder in your home folder

1
  mkdir ~/node_modules

Add symbolic link to the ‘node_modules’ folder you just created inside your project folder

1
  ln -sf ~/node_modules /vagrant/your-project-folder

This solution will ensure that all the node modules are stored inside home directory instead of synced folder.


Enable HTTP authentication on certain domain

Basic HTTP authentication is one simple way to limit public access to your website prior to launch.

The first thing you need is .htaccess file which contains all the configurations. The second one is .htpasswd containing username and password. You can use this website to generate .htpasswd file for you http://www.htaccesstools.com/htpasswd-generator/

In the sample below, I am trying to enable HTTP authentication only on certain domain. On the first line, I set enviroment variable if the domain name is equal to “www.bundabergfestival.com.au”. On line 7, I tell .htaccess file to deny any access by using the live_uri variable. I hope that explanation is pretty straight forward.

1
2
3
4
5
6
7
8
9
10
SetEnvIf Host "^www.bundabergfestival.com.au" live_uri
AuthName "Bundaberg Festival Website Coming Soon"
AuthType Basic
AuthUserFile /var/app/.htpasswd
AuthGroupFile /dev/null
require valid-user
Order allow,deny
Allow from all
Deny from env=live_uri
Satisfy any

AngularJS Free Course by Code School

I spent some time last weekend to run through the new AngularJS course by Code School. This course is sponsored by Google which means you don’t need to pay for the Code School membership to play this course. AngularJS is one of the popular Javascript frameworks to build single page app application. You might be familiar with other frameworks such as Ember.js, Backbone.js and Knockout.js.

I really enjoyed the course and definitely learned something new from it. It covers the basic concept about directives, services and dependecy injection. I didn’t understand those features when the first time I learned about AngularJS. In the beginning of learning AngularJS, I tended to copy and paste code without understanding the meaning behind it. This caused confusion when I tried to learn more about the framework.

Code School also released another screencast on how to build AngularJS app from scratch. I’ll suggest you to check that one out as well so you can apply the knowledge that you have learned from the course to build real application. However, you have to become the member to get access to the screencast. There are also some other websites that provide AngularJS videos such as http://www.egghead.io and http://www.thinkster.io


Configuring Elastic Beanstalk Environment with .ebextensions

At Captiv8, we are using Amazon AWS to host most of our PHP projects. We are heavily rely on Elastic Beanstalk to help us set up PHP environment, database and load balancer. On top of that, we are also managing our own server image based on Amazon AMI. In this image, we installed additional software and packages that we need for our application. However, this approach has a drawback as it is difficult to maintain the image and track changes. Everytime you need to update the image, you have to create new server, install the new software and export it into new image. This will leave you with bunch of different images and it is hard to tell what are the things that have changed inside each image.

Chef

In order to solve these problems, I decided to find a way to automate the process. My first attempt was trying to use Chef to provision the Elastic Beanstalk environment. I have been using Chef for a while to provision my Vagrant machines. It is powerful and more convenient in compare with bash scripts. Since I am already familiar with Chef, I started looking at tutorials on how to use Chef with Elastic Beanstalk. Most of the tutorials that I found don’t provide easy way to integrate Chef with Elastic Beanstalk. One of them mentions about using AWS OpsWorks with Chef but I think it is overkill for the time being. So, I ditched Chef and start looking for another solution.

ebextensions

ebextensions is another solution that I found after checking the official documentation of Elastic Beanstalk. With this solution, you need to to create .ebextensions folder inside your project and create a file to define what are the packages that you want to install into the environment. Elastic Beanstalk will automatically run the script every time you deploy a new version of the application. Aside from that, you can also tell ebextensions to execute shell script in the instance or changing permission of a file. You can read more details about ebextension here: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html

This is my directory structure at the moment:

Here is the example of my ebextension config file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
packages:
  yum:
    mlocate: []

commands:
  01updateComposer:
    command: export COMPOSER_HOME=/root && /usr/bin/composer.phar self-update
  02updateTag:
    command: ec2-create-tags $(ec2-metadata -i | cut -d ' ' -f2) --tag Project=ChangeThis
    cwd: /home/ec2-user
    env:
      EC2_HOME: /opt/aws/apitools/ec2
      EC2_URL: https://ec2.ap-southeast-2.amazonaws.com
      JAVA_HOME: /usr/lib/jvm/jre
      PATH: /bin:/usr/bin:/opt/aws/bin/

container_commands:
  01-command:
    command: updatedb
  02-command:
    command: rm -rf /captiv8/.ebextensions
  03-command:
    command: mkdir -p /captiv8/.ebextensions
  04-command:
    command: cp -R .ebextensions/* /captiv8/.ebextensions/
  05-command:
    command: bash /captiv8/.ebextensions/scripts/app-setup.sh

option_settings:
  - namespace: aws:elasticbeanstalk:application:environment
    option_name: COMPOSER_HOME
    value: /root

And this is the example of my bash script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
#!/usr/bin/env bash

#
# References: 
# - http://www.hudku.com/blog/configuration-setup-customizing-aws-elastic-beanstalk/
# - http://www.hudku.com/blog/security-credentials-customizing-aws/#.elastic-beanstalk-app
#

# Main configuration, change these for each project
appName="change_this"
newrelicLicense="newreliclicense"

# Check if this is the very first time that this script is running
if ([ ! -f /root/.not-a-new-instance.txt ]) then
  newEC2Instance=true
fi

# Install applications if this is new instance
if ([ $newEC2Instance ]) then
    # Allow sudo command to be used as part of beanstalk ebextensions scripts without a terminal
    grep -q 'Defaults:root !requiretty' /etc/sudoers.d/$appName || echo -e 'Defaults:root !requirettyn' > /etc/sudoers.d/$appName
    chmod 440 /etc/sudoers.d/$appName

    # Add sudo command if not already present to .bashrc of ec2-user so that we are logged on as root when we use ssh
    grep -q "sudo -s" /home/ec2-user/.bashrc || echo -e "nsudo -sn" >> /home/ec2-user/.bashrc

  # Install phpMyAdmin
  yum -y --enablerepo=epel install phpmyadmin
  rm /etc/httpd/conf.d/phpMyAdmin.conf
  rm /etc/phpMyAdmin/config.inc.php
  mv /captiv8/.ebextensions/templates/phpMyAdmin/phpMyAdmin.conf /etc/httpd/conf.d/
  mv /captiv8/.ebextensions/templates/phpMyAdmin/config.inc.php /etc/phpMyAdmin/
  chmod 644 /etc/httpd/conf.d/phpMyAdmin.conf
  chmod 644 /etc/phpMyAdmin/config.inc.php
  service httpd restart

  # Install New Relic
  rpm -Uvh http://yum.newrelic.com/pub/newrelic/el5/x86_64/newrelic-repo-5-3.noarch.rpm
  yum -y install newrelic-php5
  echo -ne '\n\' | newrelic-install install
  rm /etc/php.d/newrelic.ini
  mv /captiv8/.ebextensions/templates/newrelic/newrelic.ini /etc/php.d/
  chmod 644 /etc/php.d/newrelic.ini
  perl -pi -e "s/PHP Application/$appName/g" /etc/php.d/newrelic.ini
  perl -pi -e "s/newrelicLicense/$newrelicLicense/g" /etc/php.d/newrelic.ini

  # Install New Relic Server Monitor  
  yum -y install newrelic-sysmond
  nrsysmond-config --set license_key=$newrelicLicense
  /etc/init.d/newrelic-sysmond start
  service httpd restart

  # Install OSSEC
  yum -y install mysql-devel postgresql-devel
  wget http://www.ossec.net/files/ossec-hids-2.7.1.tar.gz -P /captiv8
  tar xzvf /captiv8/ossec-hids-2.7.1.tar.gz -C /captiv8
  rm /captiv8/ossec-hids-2.7.1.tar.gz
  rm /captiv8/ossec-hids-2.7.1/etc/preloaded-vars.conf
  mv /captiv8/.ebextensions/templates/ossec/preloaded-vars.conf /captiv8/ossec-hids-2.7.1/etc/
  /captiv8/ossec-hids-2.7.1/install.sh
  /var/ossec/bin/ossec-control start
fi

# If new instance, now it is not new anymore
if ([ $newEC2Instance ]) then
    echo -n "" > /root/.not-a-new-instance.txt
    chmod 644 /etc/php.d/.not-a-new-instance
fi

Inside my ebextensions config file, I call the shell script which will install additional software. The benefit of using shell script is you have more options and it is much easier to customize the software. Since the .ebextensions folder is copied to the instance, you can tell the shell script to copy a template config file that you have prepared before hand. I hope you find this blog post useful.


Audio Books

Around October 2013, I decided to move from city to Gladesville which located 9 kilometres north-west of Sydney CBD. It wasn’t easy decision because I need to spend about 2 hours on commuting every day.

For the first couple months, I was fine with the long commuting time. I can distract myself with my phone or sleeping on the bus. However, I started to feel unproductive and I can feel it affected my concentration throughout the day. So, I tried to do some research on the Internet about productive use of commuting time. Most of the articles that I found suggesting to listen either podcasts or audio books.

I decided to give audio book a try and downloaded Eat That Frog by Brian Tracy. If you are interested, you can easily find the audio version of this book on YouTube. At first, I was a little bit skeptical with the result but it turned out to be really helpful. I can feel that my life is back on track again. I started to use Trello to keep my list of tasks. Once in a while, I’ll update my goals and create separate boards for each project.

Since that, I have been listening to several different audio books such as 168 hours by Laura Vanderkam and Getting things done by David Allen. Both of them shared some basic principles from Eat that frog. I prefer Getting things done because it’s more straight forward and focusing on actions you can perform to improve your productivity. Although, I can say that this book is quite to hard understand so you need to listen it more than once to get a good grasp of the concept.


Add ‘current’ class to website menu in Rails

It is common to have ‘active’ state or ‘current’ state on website navigation. This will help visitors to know which page they have selected.

This solution is based on Stackoverflow’s question which I couldn’t find. First, I’ll create a method inside Rails application_helper.rb file. I’ll call this method cp(). Here are the syntax:

application_helper.rb
1
2
3
4
5
6
module ApplicationHelper
  def cp(path)
    current_route = Rails.application.routes.recognize_path(path)
    "current" if current_page?(path) or params[:controller] == current_route[:controller]
  end
end

The method uses current_page and Rails.application.routes.recognize_path to get information about current page.

After that we can use it in our view. Here is the example:

application.html.erb
1
2
3
4
5
6
7
8
9
<nav id="menu-panel">
    <%= link_to 'SERVICES', services_path, class: cp('/services') %>
    <%= link_to 'FACILITIES', facilities_path, class: cp('/facilities') %>
    <%= link_to 'ABOUT', about_path, class: cp('/about') %>
    <%= link_to 'CAREERS', careers_path, class: cp(careers_path) %>
    <%= link_to 'BLOG', blog_index_path, class: cp(blog_index_path) %>
    <%= link_to 'CONTACT', contact_path, class: cp('/contact') %>
    <a href="#" id="close-menu-panel"><b>CLOSE</b></a>
</nav>

I hope that helps.