General

All files/directories in *nix file systems are case sensitive.

Any file/directory starting with a period is normally invisible to directory searches.

*nix will not look in your current directory for any command, it uses the $PATH command to tell it where to look. $PATH may contain ./ in which case it will look in the current directory. See whereis

The Main Directories

These are the basic directories that you (should) have after installing any Linux / BSD distribution:

/bin/

This is where all your programs that are accessible to all users will be stored once installed.

/dev/

This is a virtual directory where your devices are 'stored.' Devfs allows Linux to list devices (hard drives, input devices, modems, sound cards, etc.) as 'files.'

/etc/

This is where you'll find all your global settings. Daemons such as ssh, telnet, and smtp/pop3 mail servers find their configuration files here. Also in /etc/ is the system's password file, group lists, user skeletons, and cron jobs.

/home/

This is the default directory where non-root users' homes are created. When you add a user, the default home directory is created as /home/username. You can change this default in the proper file in /etc/.

/lib/

This is where shared libraries (perl, python, C, etc.) are stored. Also in /lib/ are your kernel modules.

/mnt/

This is the default location for mounting cdroms, floppy disk drives, USB memory sticks, etc. You can mount anything anywhere, but by default there is a /mnt/floppy (if you have a floppy drive) and /mnt/cdrom.

/proc/

This virtual folder contains information about your system. You can view processor statistics/specifications, PCI bus information, ISA bus information, and pretty much anything else you want to know about the hardware on your system.

/root/

This is the default home directory for the user root.

/sbin/

This is where system programs are installed. These include fdisk, tools to make partitions, certain network tools, and other things that normal users shouldn't have a need for.

/tmp/

This is the default location to place files for temporary use. When you install a program, it uses /tmp/ to put files during installation that won't be needed once the program is installed. This directory is cleared when the system reboots.

/usr/

This contains various programs, non-daemon program settings and program resources.

/var/

This is where your log files, system mail messages, web sites and database of installed programs are stored.

.profile - this file is held in the users home directory and is executed when a new terminal window is started, it tends to be used to set individual choices to command search path, editors, terminal settings there is also a system wide profile file located in /etc

& - run in the background

see nohup and fg and jobs

$? - return value

A command can return a numeric value when it finishes, this can be got from $?

alias - create a new command from others

alias xx="ls -l"

creates a command called xx which is a long list command

(see .profile)

awk - reporting language -does not stand for awkward.

Great for generating reports from csv files, see http://tldp.org/LDP/abs/html/awk.html for examples

basename - return the last component of a pathname

basename myfile.txt

returns myfile

bg - background

Places the current job (or, by using the alternative form, the specified jobs) in the background, suspending its execution so that a new user prompt appears immediately. Use the jobs command to discover the identities of background jobs

cd ~ - return to my home directory

cd - - return to the last directory I was in

cp - copy a file(s)

cp file1 file2

cp file1 /var/log

cut - remove sections from lines of a file

This could be used to remove columns from a csv file

cut -f1,3,5 file.csv

output only the columns 1, 3 and 5 from the file

egrep – extended grep

As fgrep, but allows reg exp to be used, and is therefore slower

egrep ‘HTML|BODY|HEAD’ index.html

Only output the lines that contain, HTML or BODY or HEAD

fg - bring any background jobs/commands back to the foreground

fgrep – fast grep

this function does not allow reg exp patterns to be searched for.

fgrep -r ‘main()’ *

recurse through the directory structure looking for main() in any file it finds

fgrep -r ‘getCustomerById(‘ * | fgrep -v svn

search all files for this string, then remove any references to subversion files

find - search for files in a directory structure

This is a very powerful command with many options.

find $HOME -name 'mysong.ogg'

look for a file called mysong.ogg in the home directory and subdirectories

find $HOME -iname '*.ogg'

search – ignore case – for any ogg file.

find $HOME -iname '*.ogg' -size +100M

search for any ogg files greater than 100Mb

find $HOME -atime +30

look for any files that has been accessed in the last 30 days

find $HOME -iname '*.ogg' -size +100M -exec rm {}

this will find any ogg file greater than 100Mb and then delete them

find -r ‘INSERT INTO’ * | fgrep -v ‘svn’ | wc -l

do a recursive search for any file containing ‘INSERT NTO’ , ignore all files that have svn in their name and then

count the number of lines

head - like tail, but display the top x lines of a file

history – what you have done at the command line.

This will give a numbered list of commands, you can repeat a command by typing ! xxx, (no space between ! and xxx) where xxx is the number of the command you wish to run

history | fgrep ssh

report all the times I used the ssh command.

hostname - is used to display the system's DNS name, and to display or set its hostname or NIS domain name.

kill - stop a process from

this requires the pid ( process id ) to be killed

less - page the output from a file

cat /etc/hosts | less

ln - link files

ln -s /home/simon/demo /home/jules/mylink3 #Create mylink3 pointing to demo

locate - find a file anywhere within the file system

man - manual pages

man less

man cp

man fgrep

mount – show all mounted devices or mount a new one

a device – usb drive, camera – is mounted – usually into the /mnt directory or /media – to allow you to access it.

mv - move or rename a file

mv fred bert

renames the file fred to bert

mv fred /var/log

move the file fred to the directory /var/log and call it fred

nice / renice – be nice to all processors that are running

nice takes a value from -19 to +20, most processors start life with a value of between 0 and -5 (the lower the number the less nice you are to other processors and the more system resources your process gets).

nohup - no hang up

nohup cmd

run the cmd and do not kill it when I log out. The cmd will run until it terminates, not because you have logged out or killed the terminal window. Any output will be redirected to a file called nohup.out in your home directory

nohup wget http://www.example.com

can also be used as

nohup wget http://www.exampe.com &

this last character runs the command in the background (see fg) and will return a process id (pid)

tail - display the last 10 lines of a file

tail /var/log/apache2/error.log will display the last 10 lines of this file

tail -50 /var/log/apache2/error.log will display the last 50 lines of this file

tail -f /var/log/apache2/error.log will keep displaying the last line of the file as it gets written to

time - run a command and summarize the system resource usage

time wc /etc/hosts

This will report three values, Real time, User time and System time.

top - show what processors are running and what the machine is doing

touch - change the date timestamp of a file(s) to the current date and time

touch -t 201201311759.30 (year 2012 January day 31 time 17:59:30)

ps - list the processors that are currently running

ps -ef

list all processors

ps -ef | fgrep http

list any apache processors that are currently running

sed - stream editor

this allows you to repeat the same editing commands to a set of files.

sed ‘s/]*>//g’ index.html

remove all tags from this file

Check this page out http://tldp.org/LDP/abs/html/x22444.html to see the full power of sed

sort - sort a text file

sort y

sort the file x in alphabetic order and write the results to file y

sort -u y

sort the file, but only write the unique record to file y

split - Splits a file into several smaller files.

Can be used to split a large file into smaller chapters or number of lines.

wc - word count

Print newline, word, and byte counts for each FILE

cat /etc/hosts | wc

Count the number of words in a file

cat /etc/hosts | wc -l

Count the number of lines in a file

whereis - Report all known instances of a command

This will give a list of paths to where the command can be found. This can be useful when there is more than one version of a command on the system.

uname - Print certain system information.

uname -a

Will display all the system information:

Linux red 2.6.35-27-generic #48-Ubuntu SMP Tue Feb 22 20:25:29 UTC 2011 i686 GNU/Linux

Notes

| (pipe) allows you to pipe the output from one command into the input of another

cat /etc/hosts | wc -l

(greater than) write the output from a command into a file, if the file exists then truncate it first

fgrep -r ‘SELECT ‘ * > ~/x

append to an existing file

fgrep -r ‘REPLACE ‘ * > ~/x

fgrep -r ‘INSERT ‘ * >> ~/x

This example is meant to show how complex piping can get

sed 's/>n|<CONNUM|<URN' | awk 'END{print RS} $0=(//?NR==1?_:RS:FS)$0' ORS=

1. file.xml is a single line xml file

2. edit it to substitute all ’ > newline or <CONNUM or <URN

3. now put the multi lines back into one line per set, terminating on the tag

superKill.sh – bash shell script to kill all occurrences of a program running

!/bin/bash

for i in `ps -ef | fgrep "$1″ | fgrep -v 'fgrep' | tr -s " " | cut -d " " -f2`

do

kill $i

done

sudo superKill.sh apache

the back ticks on the command `ps -ef | fgrep "$1″ | fgrep -v 'fgrep' | tr -s " " | cut -d " " -f2` allow it to be executed as one step within this for loop. ‘i’ takes each value in turn and presents it to the kill command.

1. ps -ef ← list all processors with full formatting

2. fgrep “$1” ← search each line for the command line parameter

3. fgrep -v fgrep ← remove any calls to the actual search

4. tr -s “ “ ← convert any multiple spaces to a single space

5. cut -d “ “ -f2 ← with a delimiter of space cut out the second field

This will return all the pids for the given argument

the above could also be run as

kill `ps -ef | fgrep "svn" | fgrep -v 'fgrep' | tr -s " " | cut -d " " -f2`

Redirection

Under normal circumstances every *nix program has three streams opened for it when it starts up, one for input, one for output, and one for printing diagnostic or error messages.

These are typically attached to the user's terminal, but might instead refer to files or other devices, depending on what the parent process chose to set up. They are referred to as stdin (0), stdout (1), and stderr (2).

Before a command is executed, its input and output may be redirected using a special notation interpreted by the shell.

command 1> out.txt 2>err.txt – may create two txt files ( if any errors occur)

Note that the order of redirections is significant. For example, the command

ls > dirlist 2>&1

directs both standard output and standard error to the file dirlist, while the command

ls 2>&1 > dirlist

directs only the standard output to file dirlist, because the standard error was duplicated as standard output before the standard output was redirected to dirlis

Quotes

There are three types of quotes in the BASH shell ( terminal) , single ‘, double “ and back or ticks ` , and they all do something different. The first two work as they do in PHP with or without substitution. The last one – ticks – allows you to run the string as a command and return any values.