All posts in Unix and Linux

My review after two months using VIM

On January ( Two months ago ) , I decided to learn Vim after watching screencast from Destroy All Software. In one of the screencast, the guy shows how fast you can program when using Vim and use plugin to make it even better.

I have known Vim for 4 years now but never actually keen to learn all the shortcuts. I remember I been to one of the PHP meetup in my city and one guy was presenting some of his code using Vim. One of the audience commented “Why you need to use Vim if you have IDE and all of the GUI ?”. At that time, I couldn’t agree more with that guy.

I had always thought Vim’s user are just trying to show off that they are different and superior. Using Vim basically separate them from GUI’s group and put them one level above. At least, that’s what I was thinking and apparently I am wrong.

After started using Vim, I can feel that I am more productive and less stressful when programming. All the shortcuts definitively help you to avoid redundancy and remove your reliance on mouse. You can navigate through the source code by only using keyboard.

Practical Vim

The book that I am using to learn VIM is “Practical Vim : Edit Text at the Speed of Thought” by Drew Neil. If you are new with Vim, I’ll strongly recommend to start with this book. This book has collection of tips on how to use Vim in real situation and you going to feel you learn Vim faster.

However, before deciding to learn Vim, you need to know that Vim is not for everyone. You know that you are not suitable for Vim when you :

  • Still looking at your keyboard when typing.
  • Have problem or hate remembering shortcuts.
  • Don’t have any curiosity to find another way to achieve something.

If you are an IDE fan such as Eclipse, Netbeans or Visual Studio. There are some plugins that you can download to unleash the Vim power in your IDE. At the moment, I am using VsVim to use all the Vim shortcuts when programming in C#. If you want to become better programmer, I’ll say start using Vim and you won’t regret it.

Precise Pangolin

Screenshot from 2012-07-31 17:31:53

After spending whole day moving all my data from Ubuntu partition, I am officially running Ubuntu 12.04 64 bit. Prior to this, I was using Ubuntu 12.04 32 bit which I continuously upgrade from Ubuntu 10.04 ( it’s been 2 years since I decided to use Ubuntu as my main operating system, time flew so fast ). It was running really slow as I have a lot of applications installed ( Rails, Django, Android SDK, Eclipse, Netbeans, etc ).

Basically I am happy with my new “fresh” installation of Ubuntu. Everything seems to run 3-4 faster than before. Booting time only takes around 20 secs from 1-2 minutes, Sublime Text fired up as soon as I clicked it and Chrome jumping smoothly like kangaroo. However, I just realize that I forgot to back up my Thunderbird email account ( damn!). Before, I set up my Thunderbird email account in POP which means that it will delete all the emails in the server. So now I am losing 17 months of emails archive which I have to explain to my boss soon.

So here are my applications line up for my “fresh” OS :

  1. Email Client : Thunderbird.
  2. IDE Text Editor : Sublime Text 2 and Vim.
  3. Web Server : Apache ( I was considering using Nginx, however it will take too much time rewriting all the .htaccess rule into Nginx configuration. I can’t afford that ).
  4. Web Browser : Chrome.
  5. Source Control : Git and SVN.
  6. Music Player : Rhythm box ( stay with default ).
  7. Coming soon…

Next stop ? Installing Windows 7 64 bit and VirtualBox.

Assembla

Recently, I just canceled my Github micro plan and moved all my private projects to Assembla. Similar to Github, Assembla also offers Git hosting and you can host private projects for free. There are several paid plans that you can choose which basically add more features such as ticketing system, wiki, FTP build, supports, etc. So if you are like me ( only need Git/SVN private hosting for your projects ), I suggest you to try Assembla.

Another great thing is no limitation of number of users that you can give access to your private projects. While in Github, the micro plan has limitation of one user that you can allow to access your projects.

There is not much I can comment about the UI as these two websites completely go in different direction. Github enhances social networking between programmers by allowing users to watch public project, fork and send pull request. This feature is very useful for open source projects to attract more contributions.

On the other side, Assembla does not seem to focus on public spaces. I can say that because I tried to search for PHP public project before and the search page returned “Internal Server Error”. Also, Assembla puts all your projects on the menu bar but only for three projects. You have to click more spaces and it will show up more projects that you have. This approach will make your life difficult if you have more than 10 projects.

Finally, I can say that I am happy with Assembla for the moment but I’ll back to Github when I have bigger team and more open source projects to manage.

Gource

Gource is a software version control visualization tool. It was built using C/C++ ( not sure ). Several version controls that supported by Gource are Git, SVN, Mercurial and Bazaar.

Basically Gource is a command line tool that you can use to visualize your project’s progress that using software version control. Here is the example of my project :

Cool isn’t it ? In my video, there is only one person because I am working for this project alone. However, if you are looking in YouTube, you can find a lot of big projects that using Gource to visualize their progress. One of the project that you should see is the Linux Kernel project.

Here is the link to the Gource Website

Apache GeoIP

Apache GeoIP is an apache module for detecting your visitor’s IP and block/redirect them to another website. Installing and configuring GeoIP is very easy in Ubuntu. Here are the steps :

1.Install the Apache GeoIP module and restart apache webserver

sudo apt-get install libapache2-geoip
sudo /etc/init.d/apache2 restart

2.Create the .htaccess file and put in your webroot folder / website folder

GeoIPEnable On

# Redirect one country
RewriteEngine on
RewriteCond %{ENV:GEOIP_COUNTRY_CODE} ^AU$
RewriteRule ^(.*)$ http://www.canada.com$1 [L]

In the example above, GeoIP will check whether the visitor is from Australia or not. If the visitor is from Australia then apache will redirect the visitor to http://www.canada.com. Quite simple and straight forward. You can see another example in here http://www.maxmind.com/app/mod_geoip ( ignore the GeoIPDBFile because it will cause error ).

Redcar : Textmate alike editor

It’s been a long time since the last time I am using e-texteditor as my primary text editor ( e-texteditor is a textmate alike editor for Windows ). And because I decided to use Linux, I have to find another editor and now I ended up using Eclipse. In Linux, obviously, there are several text editor but the features doesn’t fit my needs.

But now there is a new text editor that written in Ruby and runs on JRuby and it’s features are inspired by Textmate : Redcar. Actually, I already tried to use redcar last year and at that time the features are not as complete as now.

You can follow the development through it’s github repository. The author Dan, is very active and looks like he commit often :)

As far as I tested, the bundles support is good enough in compare with e-texteditor ( I am using version 0.5.1 ) and bundles is the top reason why I like using Textmate-alike editor. Looking forward to use it as my primary text editor in Linux.

Add bridge and assign unique IP for each container in OpenVZ

I am writing this tutorial to refresh my mind ( I am not configuring the machine right now ), so forgive me if the explanations are not clear and structured.

After successfully installed the OpenVZ, you need to configure the IP for the containers. If you are using the built in “Venet” you will end up making the host as the gateway. In the other words if you traceroute to your container IP, it will going through your host then to your container. Here are the sample of the scenario :

  1. Host IP : 202.58.181.2
  2. Container 1 : 202.58.181.50
  3. Container 2 : 202.58.181.51

Basically if you use the venet, if you traceroute to the Container 1 the result should be : Your ISP -> 202.58.181.2 -> 202.58.181.50 so the IP of the Host is showed up. Same if you try to traceroute to the second container. Thus, my solution to overcome this problem are :

  1. Make a bridge using brctl command
  2. Use the Veth instead of Venet
  3. Put all the Veth into the bridge
  4. Assign the Host IP to the bridge

If you are not ASSIGN the host IP to the bridge, you can’t access the host machine. I followed the guidelines from this link http://wiki.openvz.org/Using_veth_and_brctl_for_protecting_HN_and_saving_IP_addresses

Here is my /etc/rc.local configuration looks like :

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
brctl addbr vzbr0
brctl addif vzbr0 eth0
brctl addif vzbr0 veth1.0
brctl addif vzbr0 veth2.0
brctl addif vzbr0 veth3.0
brctl addif vzbr0 veth4.0
ifconfig vzbr0 0
ip addr add 202.58.181.2/25 dev vzbr0
cd /usr/local/webvz/ && /usr/bin/ruby script/server &>/dev/null &

I’ll update this post if I need to configure another VPS Server.

Installing OpenVZ and WebVZ in Centos 5.5

OpenVZ is an operating system-level virtualization technology based on the Linux kernel and operating system. OpenVZ allows a physical server to run multiple isolated operating system instances, known as containers, Virtual Private Servers (VPSs), or Virtual Environments (VEs).

Now we’ll try to install OpenVZ in Centos. OpenVS is an operating system-level VPS so you need to install Host Operating System ( if you are using hardware virtualization like Xen or VMware then you don’t have to install an Operating System ).

The first step is that you must install the Centos ( I am using Centos 5.5 ). I won’t explain about how to install it so you must find the tutorial yourself :P

Let’s say you already installed the Centos, next step is updating your yum package manager. Here are the steps :

  • Download the OpenVZ repo from http://download.openvz.org/openvz.repo
  • Put it in /etc/yum.repos.d directory
  • Run the yum update command

Wait until the process is complete, it will downloading all the necessary update package. ( you can find the complete tutorial in OpenVZ Wiki ).

Now we will install the OpenVZ, type this to install the OpenVZ :

yum install ovzkernel vzctl

If the installation running smooth you will have OpenVZ installed, reboot your machine :

reboot

Press any key when the Grub screen is popping out. If it’s installed then you must have 3 kernels. Choose the top kernel ( it’s the OpenVZ kernel, in my machine the name is CentOS ( 2.6.18-194.3.1.el5.028stab069.6 ).

Log in to your machine as usual, now you are running on OpenVZ kernel. The next step is we will install WebVZ ( the OpenVZ management GUI ). The WebVZ, like it’s name, is a web application that runs on Ruby on Rails. So you must have Ruby on Rails in your machine to use the WebVZ.

To help us installing the Ruby on Rails we will use Ruby Works . Ruby Works is one of the CentOS repository that provides with the newest installer of Ruby, Gem and Rails.

Here are the steps :
1.Download the repo file

wget http://rubyworks.rubyforge.org/RubyWorks.repo

2.Put it the file in the /etc/yum.repos.d
3.yum update

yum update

4.Installing the ruby packages

yum install ruby ruby-devel ruby-irb ruby-rdoc ruby-ri

5.Check the installed Ruby using this command ( it should print out the ruby version eg: ruby 1.8.6 )

ruby -v

6.Then download the latest gem at Rubyforge
7.After download the latest gem, extract it, go the folder and run the installer

ruby setup.rb

8.Check if the gem is installed successfully ( it should return the version too )

gem -v

9.Run the gem update to make sure the gem is up to date.

gem update

10.Install the rails

gem install rails -v=2.3.2

11.The WebVZ need sqlite to run, so you must install the sqlite and the gcc compiler

yum install sqlite-devel
yum install gcc make

12.And the last is installing the sqlite for ruby

gem install sqlite3-ruby -v=1.2.5

13.Download the WebVZ installer from it’s website. Extract it and run the application.

cd webvz
ruby script/server

That’s all, then you can access it via web browser http://your-ip:3000. the username is admin and the password is admin123.

Update : ( run the WebVZ as daemon )
to run WebVZ as daemon just go to it’s directory run this command instead of plain “ruby script/server”

ruby script/server &>/dev/null &

and to make it’s run everytime you boot the machine ( put it in rc.local )

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

cd /usr/local/webvz/ && /usr/bin/ruby script/server &>/dev/null &

exit 0

Updating ports in FreeBSD 8.0

Ports is the applications repository in FreeBSD. Unlike Ubuntu that provides you with command “apt-get”, in FreeBSD you must manually go to the application folder in “/usr/ports” ( the folder where Ports remain ) and search for the application that you want to install. But what’s the differences between you compile it yourself and using ports. The answer is “dependencies”.

Enough with the introduction, let’s go the main topic. So we are going to update the Ports using cvsup. Yeah… yeah… I know that there are several ways to update the ports, but I prefer to use cvsup ( because it’s the way I taught ). The steps are :
1.Install the cvsup

pkg_add -vr cvsup-without-gui

2.Copy the ports-supfile from example

cp /usr/share/examples/cvsup/ports-supfile .

3.Rehash

rehash

4.Edit the ports-supfile, find the line that say CHANGETHIS.FreeBSD.org. Change it with the mirror that is near you ( I am using BizNet )

5.Run the ports-supfile using cvsup

cvsup -g -L 2 ports-supfile

Wait until the process is complete and your ports will be updated.

Copying files between 2 unix servers

There are many solutions to copy files between 2 unix servers, such as :

  • Copying the files from unix server to your local machine, and then upload the files to the destination server.
  • Using NFS to share the files from your source server to your destination server.
  • Using FTP to copy the files.

First solution is recommended if your files are not much and small in sizes. But if your files are big, then you must provide big storage in your local machine. Using NFS can be an alternate solution, it’s easy to set up, easy to mount the directory and easy to copy the files. But sometimes there are some issues with the NFS, so you must be careful in using NFS.

The last solution is using FTP. Maybe you are wondering “How can I copy between 2 unix servers using FTP ? Do I have to copy it to local machine first ? “. Actually you don’t have to, because there is a technology called ‘XFTP’. They said that you can open 2 Unix Server in different window and copy between that two machines. But I haven’t tested that yet.

Another way is using “self made script” in which I don’t create it myself, I copied it from here. The script allows you to connect to a ftp server, get all the the content of that ftp server and it’s all in terminal ( no GUI, no drag and drop, just bash command ). The script also provides you with several options ( you can get the content of specific folder in the ftp server ).

Here is the complete script in case the website is down :

#! /usr/bin/ksh


#
#  HardFeed  --  Harvest A Remote Directory via
#                Ftp Enhanced Exhaustive Duplication
#
#  Perderabo  11-23-02

VERSION="1.1"    # 03-16-04

USAGE="\
HardFeed [ -v | -s | -d | -r | -f | -m | -p password-file 
           -l list-command | -x ftp-command ...  ] system user [directory]

use \"HardFeed -h\" use for more documentation

-v (verbose)  Print stuff during run
-s (symlink)  Attempt to duplicate any remote symlinks
-d (directory)Attempt to duplicate any remote directories
-r (recurse)  Attempt to descend into any directories and process them
-f (freshen)  If remote file is newer than local file, overwrite the local 
-m (mode)     Attempt a chmod on files and directories that we create
-p (password) Specify a file that contains the password in plaintext
-x (extra)    Specify a command to be sent to the ftp client 
-l (listcmd)  Override the choice of \"ls .\" to get a remote directory"

DOCUMENTATION="HardFeed  Version $VERSION

$USAGE

HardFeed copies all of the contents of a remote directory to the current 
directory using ftp.  It establishes an ftp connection to the remote 
site and it uses the ftp command \"ls\" to get a listing of the remote
directory.  The two required parameters are the remote sytstem and the user
name.  The optional third parameter is the remote directory to copy.  The 
default is just the home directory of the ftp account.

HardFeed will prompt you for the password.  This is very secure but it isn't
any good if you want to run HardFeed automatically. You can set the password in
the environment variable HARDFEED_P as an alternate.  HardFeed will set an
internal variable to the password and then clobber the variable HARDFEED_P,
since on some systems, the environment of another process can be displayed.
With most shells, you can also set an environment variable for one command
only, like this: \"HARDFEED_P=xyzzy HardFeed -dR ftpxy joeblow sourcedir\". 
A second alternative is to specify a \"password file\" with the -p option.  
Such a file contains, in plaintext, the password.  HardFeed will read the file
to get the password.  You must decide which option makes more sense in your
environment.

Only files are examined.  If we don't have a copy of the remote file, we 
will get it.  HardFeed will never overwrite an existing file system object
with one exception.  If you specify -f and we have both a remote file and a
local file, the timestamps are compared.  If the remote file is newer, a
retrieval attempt will be made.  The local file must be writable for this
to succeed.  For the timestamp compare to work, you and the remote system
must be in the same timezone.  (You can vary your environment to make this
true.)

Normally symbolic links are ignored. But with -s, we will attempt to create
a symlink with the same link data.  Even with -s, we will never overwrite
any existing object with a new symbolic link.  You will need to review any
symlinks created and probably correct them.  

Normally, directories are ignored.  If you specify -d, HardFeed will attempt
to create the directory locally.  But again, it will never overwrite an
existing object to create a directory.  If you specify -r, HardFeed will 
attempt to recurse into a directory and process all of the files there.  If
you use both -d and -r, it will copy an entire directory hierarchy.  But you
can leave off -d and only pre-create a few directories if you want.

HardFeed will attempt a chmod of any file or directory that it creates if you 
specify -m.  It will try to match the mode of the remote object.

HardFeed operates by establishing a co-process to the ftp command.  Normally,
the output from the co-process is sent to an un-named file in /tmp and
discarded.  If you want to capture this output, connect a file to fd 3 and
HardFeed will use it for this purpose.  From ksh the syntax is 3>file.  You 
can also do 3>&1 to see it real time during the run if you really want.

You can make HardFeed send the ftp co-process some extra commands after the
connection is established with -x.  

HardFeed gets a directory listing by sending a \"ls .\" command to the server.
Some servers will list dot files with this while others won't.  You can use the
-l option to change the command if your server needs a different one to do want
you want. -l \"ls -al\" is one example that I got to work with unix.

For a microsoft ftp server, I had some luck with:
 -l \"ls -la\" -x \"quote site dirstyle\"
Note that everything is transferred in binary mode.  -x ascii will switch
everything to ascii mode.  HardFeed supports embedded spaces in filenames.  User
names may be long and contain slashes. All of this may make it somewhat usable 
with microsoft ftp servers."


IFS="" 

#
#  If the password is coming in via the environment, save it in
#  a local variable and then clobber the environment variable

unset PASSWORD
if [[ -n $HARDFEED_P ]] ; then
	PASSWORD="$HARDFEED-P"
	HARDFEED_P='********'
fi


#
#  Parse Command Line
#
set -A OPT_CMDS_LIST
OPT_DIRCMD="ls ."
OPT_VERBOSE=0
OPT_SYMLINKS=0
OPT_DIRECTORIES=0
OPT_RECURS=0
OPT_FRESHEN=0
OPT_MODE=0
OPT_PASSWORDFILE=""
OPT_CMDS=0
error=0
while getopts :vsdrfmhp:x:l:  o ; do
	case $o in
	v)	OPT_VERBOSE=1
		;;
        s)      OPT_SYMLINKS=1
		;;
        d)      OPT_DIRECTORIES=1
                ;;
	r)      OPT_RECURS=1
		;;
	f)	OPT_FRESHEN=1
		;;
	m)	OPT_MODE=1
		;;
	h)	echo "$DOCUMENTATION"
		exit 0
		;;
	p)	OPT_PASSWORDFILE=$OPTARG
		if [[ ! -f $OPT_PASSWORDFILE ]] ; then
			echo error $OPT_PASSWORDFILE is not a file
			error=1
		fi
		;;
	x)	OPT_CMDS_LIST[OPT_CMDS]="$OPTARG"
		((OPT_CMDS=OPT_CMDS+1))
		;;
	l)	OPT_DIRCMD="$OPTARG"
		;;
	?)      print error argument $OPTARG is illegal
		error=1
		;;
	esac
done
shift OPTIND-1
if ((error)) ; then
	echo "$USAGE"
	exit 1
fi
if [[ $# -ne 2 && $# -ne 3 ]] ; then
	echo "$USAGE"
	exit 1
fi
SYSTEM=$1
USER=$2
DIRECTORY=$3
[[ -z $DIRECTORY ]] && DIRECTORY=.

#
#  Read password file if one is supplied

if [[ -n $OPT_PASSWORDFILE ]] ; then
	read PASSWORD < $OPT_PASSWORDFILE
fi


#
#  Request password if it didn't come in via env or file

if [[ -z $PASSWORD ]] ; then
	print -n password -
	stty -echo
	read PASSWORD
	echo
	stty echo
fi

#
#  FD 3 will be the transcript of the ftp co-process.  If the user
#  supplied a file for this, we will use that.  Otherwise it will go
#  to a nameless file in /tmp

if print -u3 " Transcript of the ftp co-process for HardFeed" 2>/dev/null ; then
	LOGFILE=""
else
	LOGFILE=/tmp/HardFeed.log.$$
	exec 3>$LOGFILE
	rm $LOGFILE
fi

#
#  Max time to wait for arrivial of file.  This is a long time.  During
#  an interactive run, the user can use SIGINT if it seems to be taking
#  too long.  This max is intended to assure that a cron job will not
#  hang forever.

OPT_MAXWAIT=15
TIMEOUT=/tmp/HardFeed.timeout.$$

#
#  Various other initializations

LEV=0
date "+%Y %m" | IFS=" " read THISYEAR THISMONTH
((LASTYEAR=THISYEAR-1))
STARTPATH=$(pwd)
set -A DIR_FILE_NAME
set -A DIR_LINE_NUM

#
#  Function to convert month to numeric

conv_month() {
	typeset -l month
	month=$1
	case $month in
	jan)	nmonth=1  ;;
	feb)	nmonth=2  ;;
	mar)	nmonth=3  ;;
	apr)	nmonth=4  ;;
	may)	nmonth=5  ;;
	jun)	nmonth=6  ;;
	jul)	nmonth=7  ;;
	aug)	nmonth=8  ;;
	sep)	nmonth=9  ;;
	oct)	nmonth=10 ;;
	nov)	nmonth=11 ;;
	dec)	nmonth=12 ;;
	*)	nmonth=0  ;;
	esac
	echo $nmonth
	return $((!nmonth))
}


#
# Function to determine if a file system object exists
#
# neither -a nor -e is really portable  8(
  
exists() {
	[[ -f $1 || -d $1 || -L $1 || -p $1 || -S $1 || -b $1 || -c $1 ]]
	return $?
	}


#
# Function to wait for a file to arrive

waitfor() {
	wanted=$1
	if ((OPT_MAXWAIT)) ; then
		((GIVEUP=SECONDS+OPT_MAXWAIT))
	else
		GIVEUP="-1"
	fi

	while 	[[ ! -f $wanted && $SECONDS -lt $GIVEUP ]] ; do
		sleep 1
	done
	if [[ ! -f $wanted ]] ; then
		echo "FATAL ERROR:" timed out waiting for:  2>&1
		echo "            " "$wanted"               2>&1
		echo 
		print -p bye  2>/dev/null
		exit 2
	fi
	return 0
}
#
#  Function to decode an "ls -l" line.

lsdcode() {

	typeset -Z2 nmonth day
	typeset -i8 octal

	#
	#  get the line, get the first character, split line into words

	line="$1"
	char1=${line%%${line#?}}
	IFS=" "
	set -A  things -- $line
	IFS=""

	#
	#  We may have a "total" line which needs to be ignored

	if [[ ${things[0]} = total ]] ; then
		set -A  lsdc --  skip 000 000000000000 x x
		return 0
	fi

	#
	#
	parser=1
	month=${things[5]}
	xmonth=$(conv_month $month)
	if  conv_month $month > /dev/null ; then
		parser=1
	else
		parser=0
	fi

	if ((parser)); then
		#
		# Strict Left to Right Parse Routine 
		#
		# Break out the fields that we want.  This technique requires
		# that the user, group, and size fields never run together and
		# so they must have at least one space between them.  But it 
		# allows some limited support of filenames with embedded spaces.

		echo "$line" | IFS=" " read permstring junk junk junk junk \
						month day swing rawname
		if [[ $char1 = l ]] ; then
			link=${rawname#*-\> }
			name=${rawname% -\>*}
		else
			name="$rawname"
			link=""
		fi
	else
		#
		# Outside to Inside Parse Routine 
		#
		# Break out the fields that we want.  This technique requires 
		# that no white space exist in the filename.  But the user, 
		# group, and size  fields may sometimes run together without 
		# causing a problem.

		echo "WARNING:" badly formatted line in directory listing for:    >&2
		echo "        " "${line}"                                         >&2
		echo "        " attempting outside-to-inside scan                 >&2
		echo                                                              >&2

		((pname=${#things[*]}-1))
		if [[ $char1 = l ]] ; then
			link=${things[pname]}
			((pname=pname-2))
		else
			link=
		fi
		permstring=${things[0]}
		name=${things[pname]}
		month=${things[pname-3]}
		day=${things[pname-2]}
		swing=${things[pname-1]}
		if  conv_month $month > /dev/null ; then
			:
		else
			echo "ERROR: " outside-to-inside scan has also failed >&2
			echo "       " giving up on:                          >&2
			echo "       " "$line"                                >&2
			echo                                                  >&2
			set -A  lsdc --  skip 000 000000000000 x x
			return 0
		fi
	fi


	#
	#  Ignore . and ..

	if [[ $name = . || $name = .. ]] ; then
			set -A  lsdc --  skip 000 000000000000 x x
			return 0
	fi

	#
	#  decode permissions  (the permission string is first word

	set -A perms -- $(print -- ${permstring#?} | sed 's/./& /g')
	extras=0
	[[ ${perms[2]} = S ]] && { ((extras=extras+4000)); perms[2]=- ; }
	[[ ${perms[2]} = s ]] && { ((extras=extras+4000)); perms[2]=x ; }
	[[ ${perms[5]} = S ]] && { ((extras=extras+2000)); perms[5]=- ; }
	[[ ${perms[5]} = s ]] && { ((extras=extras+2000)); perms[5]=x ; }
	[[ ${perms[8]} = T ]] && { ((extras=extras+1000)); perms[8]=- ; }
	[[ ${perms[8]} = t ]] && { ((extras=extras+1000)); perms[8]=x ; }

	binary=2#$(print -- ${perms[@]} | sed 's/ //g;s/-/0/g;s/[^0]/1/g')
	((octal=binary))
	result=$(echo $octal)
	result=${result#??}
	((result=result+extras))

	#
	# Decode date and time and convert it to yyyymmddhhmm

	nmonth=$(conv_month $month)
	if [[ $swing = *:* ]] ; then
		if [[ $nmonth > $THISMONTH ]] ; then
			((year=LASTYEAR))
		else
			((year=THISYEAR))
		time1=${swing%???}
		time2=${swing#???}
		time="${time1}${time2}"
		fi
        else
                year=$swing
		time="0000"
        fi

	#
	#  Output the final record

	set -A lsdc -- ${char1} ${result} ${year}${nmonth}${day}${time} ${name} ${link}
	return
}


#
#  Function to process a remote file
#  We will not overwrite and existing file unless we in "freshen" mode.
#  And unless we are in "freshen" mode, it is an error for a file to
#  pre-exist.

process_remote_file() {
	VMESS="${VMESS} is a remote file that"
	do_get=0
	if [[ -f $name ]] ; then
		VMESS="${VMESS} already exists"
		if ((OPT_FRESHEN)) ; then
			line2=$(ls -ld "$name")
			lsdcode "$line2" 
			char12=${lsdc[0]}
			mode2=${lsdc[1]}
			datestamp2=${lsdc[2]}
			name2=${lsdc[3]}
			link2=${lsdc[4]}
			if [[ $datestamp > $datestamp2 ]] ; then
				VMESS="${VMESS} but is out-of-date and"
				do_get=1
			else
				VMESS="${VMESS} and is current"
			fi
		else
			VMESS="${VMESS} and cannot be retrieved"
			echo WARNING: no get since $name exists in ${localpath} >&2
		fi
	else
		do_get=1
	fi
	if ((do_get)) ; then
		print -p get \""$name"\"
		waitfor $name
		VMESS="${VMESS} has been retrieved"
		if ((OPT_MODE)) ; then
			chmod $mode "$name"
		fi
	fi
	if (($OPT_VERBOSE)) ; then
		echo "$VMESS"
	fi
	return 0
}


#  Function to process a remote directory
#  To this function, a remote directory is just an object that
#  may need to be duplicated in the current directory

process_remote_directory() {

	VMESS="${VMESS} is a remote directory that"
	if ((OPT_DIRECTORIES)) ; then
		if exists $name  ; then
			if [[ ! -d $name ]] ; then
				VMESS="${VMESS} cannot be created due to pre-existing object"
				echo WARNING: no mkdir since $name exists in ${localpath} >&2
			else
				VMESS="${VMESS} already exists"
			fi
		else
			mkdir "$name"
			VMESS="${VMESS} has been created locally"
			if ((OPT_MODE)) ; then
				chmod $mode "$name"
			fi
		fi
	else
		VMESS="${VMESS} has been ignored"
	fi
	if (($OPT_VERBOSE)) ; then
		echo "$VMESS"
	fi
	if ((OPT_RECURS)) ; then
		if [[ -d "$name" ]] ; then
			cd "$name"
			print -p lcd \""$name"\"
			exec 4<&-
			obtain_and_process_remote_ls "$name"
			print -p cd ..
			print -p lcd ..
			cd ..
			exec 4< ${DIR_FILE_NAME[LEV]}
			lineno=0
			while (( lineno != ${DIR_LINE_NUM[LEV]})) ; do
				read -u4 junk
				((lineno=lineno+1))
			done
		fi
	fi
	return 0
}


#
#  Function to process a remote symlink
#  Note that we deal with th symlink only --  not
#  the object (if any) that the link points to.

process_remote_symlink() {
	VMESS="${VMESS} is a remote symlink that"
	if ((OPT_SYMLINKS)) ; then
		if exists "$name" ; then
			if [[ ! -L $name ]] ; then
				VMESS="${VMESS} cannot be created due to pre-existing object"
				echo WARNING: no symlink since $name exists in ${localpath} >&2
			else
				VMESS="${VMESS} already exists"
			fi
		else
			ln -s "$link" "$name"
			VMESS="${VMESS} has been duplicated locally"
		fi
	else
		VMESS="${VMESS} has been ignored"
	fi
	if (($OPT_VERBOSE)) ; then
		echo "$VMESS"
	fi
}


#
#  If a remote object is not a file, directory, or
#  symlink, we come here.  

process_remote_weirdo() {
	VMESS="${VMESS} is a remote unknown object that has been ignored"
	return 0
	}

#
#  This function obtains an "ls" listing from the remote ftp system.  Then it 
#  scans the listing line by line to figure out what to do.  It will completely 
#  process the current directory.

obtain_and_process_remote_ls() {

	typeset rdir tmpfile okfile   ## local scope variables ##
	rdir=$1

	#
	#  Set up variables or modify them if we have recursed

	((LEV=LEV+1))
	tmpfile=/tmp/HardFeed.tp.$$.${LEV}
	okfile=/tmp/HardFeed.ok.$$.${LEV}
	if ((LEV == 1)) ; then
		localpath=$STARTPATH
		remotepath=$rdir
	else
		localpath=${localpath}/$rdir
		remotepath=${remotepath}/$rdir

	fi

	#
	#  Get a copy of the remote dir output in a local file
	#  called $tmpfile 

	print -p cd \""$rdir"\"
	print -p $OPT_DIRCMD $tmpfile
	print -p $OPT_DIRCMD $okfile
	waitfor $okfile
	DIR_FILE_NAME[LEV]=$tmpfile
	DIR_LINE_NUM[LEV]=0
	exec 4< $tmpfile

	#
	#  process each line
	#

	while read -u4 line ; do
		((DIR_LINE_NUM[LEV]=${DIR_LINE_NUM[LEV]}+1))
		lsdcode "$line" 
		char1=${lsdc[0]}
		mode=${lsdc[1]}
		datestamp=${lsdc[2]}
		name=${lsdc[3]}
		link=${lsdc[4]}
		VMESS="${remotepath}/${name}"
		case $char1 in
		skip)   ;;
		-)      process_remote_file
			;;
		d)      process_remote_directory
			;;
		l)      process_remote_symlink
			;;
		*)      process_remote_weirdo
			;;
		esac
	done 

#
#  We may have recursed...so we must put everything back the way
#  we found it

	localpath=${localpath%$rdir}
	localpath=${localpath%/}
	remotepath=${remotepath%$rdir}
	remotepath=${remotepath%/}
	rm $tmpfile
	rm $okfile
	((LEV=LEV-1))

	return 0
}


#
#  Main Program
#


ftp -inv >&3 2>&1 |&
print -p open $SYSTEM
print -p user $USER $PASSWORD
print -p binary

i=0
while ((OPT_CMDS>i)) ; do
	print -p ${OPT_CMDS_LIST[i]}
	((i=i+1))
done

obtain_and_process_remote_ls $DIRECTORY

print -p bye
wait
exit 0

You just need to copy all the script, save it to a file, upload it to your server and run it. But first you must install the ksh, in ubuntu you can just type :

sudo apt-get install ksh

Example of using it :

./hardfeed   
  • ./hardfeed : the name of the script
  • ftp server : the ip or the domain of the ftp server
  • username : the username of the ftp
  • directory : directory in the ftp server where you want to get the files from

Here the another options which I copied from the website :
HardFeed ftpserver joeblow somedir
This will connect to “ftpserver” and the user “joeblow”. It will cd to “somedir”. It will look at all of the files (and only the files) there. Each remote file that does not exist in the current directory will be copied to the current directory.

HardFeed -ds ftpserver joeblow somedir
This will work as the above example did. Except now we try to create local copies of any symbolic links or directories that we found in “somedir”. Again, though, we will not overwrite any pre-existing object.

HardFeed -rds ftpserver joeblow somedir
Now we will create copies of any remote directories and desend into them. This will copy an entire directory tree. (except that it continues to ignore special files, pipes, etc.)

HardFeed -rs ftpserver joeblow somedir
This is similiar, except we only desend into pre-existing directories. You can use this to copy part of a directory structure. Just pre-create the few directories that you want to copy.

HardFeed -rdsm ftpserver joeblow somedir
This will copy a directory tree, but this time it will try to duplicate the mode (permissions) on each remote object that is duplicated.

HardFeed -rdsf ftpserver joeblow somdir
The -f is “freshen”. Again it copies a directory tree. But if we have a local file and a remote file, the timestamps are compared. The remote file will overwrite the the local file if the remote file was newer.