UNIX/Linux notes

no guarantee

Last change: Aux 17, 2005

Related pages

Index

Colours: green is used for user's input (commands), yellow for samples of files

Commands

Notes: '|' means 'or'. Text in italic stays for any text (e.g. filename denotes any name of a file, i.e. the text to be replaced by a user)

Sections: help | Directories+files | Processes | Text utils | Network | Permissions/Access rights | X server | Other

Getting help -----------------------------------

apropos keyword - returns all the first lines of man entries that include the keyword. The first line is st. like:"xedit - simple X text editor". (But a special database must be set up.) It search a database that must be initialized (e.g. run 'root# mandb -c') and should be kept up-to-date.
man - shows a manual page for a command / utility (e.g. man ls, man gcc). See below

Directories and files --------------------------

cd directory - change dir., as in DOS (cd subdirectory | cd .. | cd absolut_path). cd without an argument enters your home directory ("cd" equals to "cd $HOME"). cd -P ../ goes to the real parent directory ignoring the symlink you used to come there.
cp source target - copy 'source' and name it as 'target'. For copying directories add '-r' flag (it ensures copying of all subdirectories etc., r means recursive):
$> cp -r ../source.directory target-directory.
ln - creates hard or soft link (i.e. a 'file' that points to another file). Use flag -s for a symbolic (soft) link.
ls - lists a content of a directory (as 'dir' in DOS). If you type ls -la more detailed information will be shown ('l' as long format and 'a' as all files).
mkdir directoryname - make directory
mv source target - moves a file from source to target
$> mv $HOME/java/file.class $HOME/cool_java // moves file.class from /java to /cool_java
pwd - prints current work directory
rm [options] name - remove file or directory named name. If removing non empty directory, then use the following:
$> rm -r directoryname . Some flags: -i asks for confirmation, -f forces the removal without asking.
shred - delete a file securely, first overwriting it to hide its contents

Processes ------------------------------------

bg/fg - brings a process to background/foreground
kill a process: 1)find it (ps -alef | grep probable_name) 2) kill it (kill -9 process_id) (note: sometimes better to use only "ps -al"; Also ps has a flag -format that let you choose what you want to see [e.g. pid&name...])
top display CPU, memory (...) and disk usage of processes (press 'q' to quit it)
fuser prints what processes are using a file/filesystem - indispensable if you can't umount/remove floppy because something i s using it and you don't know what

Text utils -----------------------------------

bhl (GNU made) converts plain TXT files into HTML, LaTeX, and SGML
cat file1 [file2 ...]
concatenates the files and prints them to the screen (->split)
conv (also dos2unix and unix2dos) convert a text file from DOS to UNIX and vice versa (EOFs). Try also iconv ($ conv -f ISO8859-2 -t WINDOWS-1250 in.txt > out.txt).
cut
  cuts a file into columns (->paste). Ex (print to the screen 2nd word (separator=space)):$> cut -d" " -f2 myfile
grep (see also agrep - a better grep)- searches specified files (if none, searches all files in the current directory) for a pattern, prints out all lines containing the pattern (+filename +line number). It's case sensitive until the flag '-i'  is used. Flag -l: print only a list of files containing >=1 matched string. Flag -n causes it to print line numbers (starting with 1). Flag -v inverts result (prints lines not containing the pattern).    $> grep -i 'mystring' *  finds & prints all lines of all files in the current pwd, containing pattern 'mystring' or 'Mystring' etc. (case insensitive). Note: with flag '-w' it doesn't match substrings (-w 'les' do not match 'lesnice'). See xargs below to see how to search recursively. $> grep -E '^a?b|c' inputfile finds all lines starting with a, ab, or c.
head displays the beginning of a file (->tail)
less - better version of 'more'. See also lesskey (lets you customize key bindings ['f' for 'forward' ...]). See the "more" command. See also lesspipe.sh: an input filter forless that allows to view files with binary content,compressed files, archives and files contained in archives
more prints a file to the screen (less is similar and better but not that common)
paste
joins files created by cut (or actually any files as long as they have the same number of lines) (->cut)
recode
changes file coding (e.g. iso-8859-2 to windows-1250). Similar but simpler is iconv, which is a part of glibc and thus shall be installed on every system.
sed - stream editor ex.: substitute (s) every line (matched by '.*' and referenced by '&') by st. else (sed s/RegExp/ByWHAT/): $> sed 's/.*/<option value="&">&<\/option>/' in.file.txt // '/' must be escaped. Ex - extract file extensions from a file list and print them, ignore all non-matching lines ( -n don't print lines by default; /p print the line): $> sed -n 's/.*\.\(.*\)/\1/p'
sort [-u] file1 [file2 ...]  concatenates the files and sorts them. Any duplicates are removed if -u is given.
split splits files into blocks of the same length
tail displays the and of a file (->head)
tee is used in a pipe to print to the standard output (the screen) what's passing through it: ... | tee | ...
tr "translates" a character (also [:space:] etc.) into another, i.e. replaces all occurrences of a character in a file by another one. Ex (dos files to unix format = delete LF?):$> tr -d '\r' < dos_file > output_unix_file. Ex2 - Change to lowercase: $> cat file | tr '[A-Z]' '[a-z]'
unique
wc file counts words, lines etc. of a file

Network --------------------------------------

hostname [i] - print computer's hostname (or IP, if the i flag is given)
scp "secure copy", similar to cp but copies between a local and a remote machine. It uses SSH (thus the remote system must run ssh daemon, i.e. sshd).
sftp secure ftp, quite the same as ftp but instead of ftp daemon it uses ssh daemon (= ssh server)
stat [-f] print status of a file - size, last acces time etc.
wget non-interactive tor for web downloading (you can start it and log off, it goes on working). wget --spider --force-html -i file.html checks links.
route - print routing tables.
netstat -pl - show what program (-pid) is listening (-l) on which port, i.e. which program opened a port. Must be run by root to display the program's name and PID.

Permissions/Access rights  -------------------

newgrp group_name changes user'a effective group to the given one. If you want to gain permissions valid for one of groups you belong to and it isn't your effective one (see id) you have to switch into it via newgrp.
chmod [who] op-code permission file - [who] may be 'o' (others), 'g' (group), 'u' (login owner), 'a' (all above). Op-code may be '+' (add permission), '-' (remove perm.), '=' (set permissions), permission may be any sub-string of string 'rwx' (r as read, w s write, x as execute). File is the name of the file, whose permission is being changed. Ex: $>chmod o=r myfile sets permission for others only to read (others can only read file myfile, not write or execute)
id prints effective user id and username as well as effective group id (gid) and its name.
groups prints what groups the user belongs to, the effective one (the one that is valid for him/her in the moment) goes first
chown, chgrp change owner/group of a file
addgroup,delgroup,adduser,deluser used by root to add/delete

X server ------------------------------------

xev - print contents of X events - use this e.g. to learn what key code is assigned to a key.
setxkbmap - changes the keyboard (what character is typed when you press a certain key). Ex - set Czech: $> setxkbmap cz_qwerty -model pc105 -option // See /etc/lib/X11/xkb/; there is a file "symbol" that contains all known layouts (cz_qwerty, us ...). See also: xmodmap (or its graphical front-end xkeycaps). Or: $> setxkbmap cz_qwerty back: $> setxkbmap us
xmodmap, xkeycaps - set a keyboard mapping. xkeycaps is a graphical front-end to it, you can see what key corresponds to what character/symbol. "$> xmodmap -pk" lists the current keymap table.
xdpyinfo - display info about the X server (current resolution, etc.)

Other ---------------------------------------

clear - clears the shell window. CTRL-l has the same effect. (CTRL stays for the Control key)
du -k directory prints the space occupied by the directory (and the same for its subdirs & files); -k = the values are in KB. (df prints file system usage info) Ex.: $> du -m --max-depth=5 | sort -nrk 1 (sort according to the first field which is a number, largest first).
echo - to print something. E.g. $> echo $MODULEPATH to see the content of varible MODULEPATH (don't forget '$' sign) To print environmental variables, you can use printenv instead (e.g. $>printenv MODULEPATH)
env - prints all variables and their values (SHELL ...).
eject eject CD-ROM
find pathname_list condition - finding a file. Pathname-list - where should it search (e.g. ~mnt/floppy, ../mydir, ./, .) Conditions - see man for complete list (e.g. -name filename, Search for more files: find . -name filen1 -o -name filen2). See the command slocate below. Flag -xdev causes not to search other filesystems (e.g. mounted in /mnt/*). Ignore dir. "dir": find -path '*/dir' -prune -o -print .
getopts is used in scripts to parse arguments from the commend line (prefixed by - or +). See a description.
less
- better version of 'more'. See also lesskey (lets you customize key bindings ['f' for 'forward' ...]). See the "more" command. See also lesspipe.sh: an input filter forless that allows to view files with binary content, compressed files, archives and files contained in archives
lp - printing (use "psnup -n 2" to fit to pages on one; takes a .ps file as input, output can be sent to printer via pipe: ... | ls)
module - module add name and module initadd name (where name referes to the module to be added, w.r.t. path specified in MODULEPATH)
more - displays its input (a file, stdin) on the screen only it is full and then waits. Press "h" in it for help. Similar but better is less.
passwd - change your password
quota -v prints the current total usage and the quota (for disc space)
rpm -iv file.rpm install a rpm package (not tested)
set, unset - change value of a variable in bash (e.g. $>setenv PATH $PATH":/student/bin"). Sometimes called setenv, unsetenv.
locate <string> or slocate - finds <string> in the database of files. Quite faster than find. The database must be initialized before.
/sbin/shutdown (-h | -r) 0 shut down / restart computer in 0 ms. See also: /sbin/halt, poweroff, reboot.
su [-] [<user>] run a new shell as the user <user>. If '-' is omitted his/her login scripts are not executed. No user = root.
sudo [ <user>] <command> - execute a command as another user (no user = root). What can be done in this way is written in the file sudoers.
test - check file types and compare values (existence, size ...) - very useful, many possibilities. Short version of this is '[ '...' ]'.
which command_name - prints PATH for the command command_name (or "command not found"). Ex: $> which java.
xargs utility [arguments] - basically takes standard input and construct a command line of the form "utility [arguments] read_from_input". Ex: $> find net/ -name "*.java" | xargs grep -n 'Network' finds all .java files in the current dir and its subdirs and then let grep search through them for the string 'Network'.

You can see a long list of programs available on my system (in Swedish).

Notes:

Good to know:

Other commands, not needed so often

man

Shows a manual page for a command / utility (e.g. man ls, man gcc). Man. pages are stored in several files and is divided into sections. and the same command may be found in more pages (e.g. function 'malloc' exists both in C and Fortran) and man usually returns only the first entry it finds (the Fortran version, in the case of malloc).  There's several options how to bypass this problem:
    1) list all matching pages (using '-l'  flag, e.g. $> man -l malloc) and the search path and section where they were found (the section is in parenthesis, e.g. (3C) for C section). Choose the one you want and use man specifying the search path by means of '-M' flag (e.g. $> man -M /usr/man malloc).
    2) let man print (show) all matching pages concatenated into one long page, by using '-a' flag ($> man -a malloc)
    3) let man search the section you want by using '-s' flag followed by the section identifier (e.g. $> man -s3c malloc where 3c, respectively. (3c) is the section containing C functions; (3C++) contains C++ func.)
    4) let man search for the keyword and print names of all found pages by the -k flag. 
Quit man by pressing 'q'. To see next screen of text press 'space', to see next line press (nearly) whatever else. 'b' to move one page up.

It can be also useful to find information on man itself ($> man man).
See the command apropos (above).

Interesting to ask man about:

(de-)compressing: tar & gzip

Content: compress, decompress.
Compress
If you need to compress or decompress files / directories, you can use utilities tar and gzip & gunzip (see www.gnu.org). Suppose you want to compress all files in a directory 'mydirectory':
$> ls                                      // you're in the parent directory of 'mydirectory'
mydirectory
$> gzip -r mydirectory    // '-r' means 'recursive' (all files in and all subdir. as well)
$> cd mydirectory          // pwd == mydirectory
$> tar -c -v -f ../a.gz.tar .                // or -cvf (f must by the last)
  '-c' stays for 'create archive', '-v' for 'verbose' (so tar will tell you what it is just doing) and finally 'f' followed by a filename specifies the output file (archive). Any existing file of that name will ber destroyed (they are some options allowing adding to the end of an existing file). Dot '.' means 'compress all files in the current directory (==pwd)'.
To tar and gzip everything in the directory DIR1 (of course you can connect them by pipe |):
$> tar -cf dir1.tar DIR1/*
$> gzip -9 tddb44.tar
Decompress:
For decompressing:
$> tar -xvf file.tar         // decompress (-x) and tell what you are just doing (v)  from the file (f) file.tar that is in the current directory. !!! 'f' mus be the last (i.e. it must be immediately followed by a filename)
$> gunzip -r mydirectory    // assume .gz files are in the directory 'mydirectory'
or $> gunzip file.gz
 

 MS-DOS files under UNIX

There exists a set of utilities for accessing DOS disks (including floppies) under UNIX. See  $> man mtools for details. Notice that you can use these tools as well to access MS Windows files.

Redirection

For more info, search "$man bash" for "REDIRECTION".

| - connects two programs via a pipe (runs the first and then the 2nd, output of the first one becomes input of the second one): $> program1 | program2
> - redirects output (standard output is a screen): $> myprogram > myoutputfile . E.g. $> cat file1 > file2  is a primitive form of copying file1 into file 2.
>> - the same as > but the output is appended into the 'file' instead of rewriting it
2> - redirects error messages
&> - redirects both standard output and error messages (&> out.txt =  >out.txt 2>&1).
< -  redirects input (standard input is a keyboard): $> myprogram > myinputfile .
& - if you run an application from a terminal it will be frozen until the appl. is finished; by using '&' at the end of the command you tell it not to wait for the appl. to finish. Ex: $> nedit &.

To set redirection for a group of commands you can enclose the commands by '{ ' and ' }':
$> { echo "cmd1"; echo "cmd2"; } > /dev/null .


Security

Root Kit
A package of hacked utilities (ls, find, ps...) and perhaps other software (kernel modules...) used to hide the activity of a hacker (his files, modification times etc.) and to help him to break in. See chkrootkit - a tool to check for rootkits on your computer.
Kernel Modules
Kernel module is a dynamically loaded part of the kernel (e.g. drivers). Since it becomes a part of the kernel it can do about anything, even change other kernel services. Thus you can use a custom module both to enhance security (a module that allows anly certain modules to be loaded) or to compromise it (a module to hide hacker's authority). So be very careful about modules you load!

Security software


Various stuff

GNOME

GNOME windows manager was Sawfish, nowadays it's Metacity (better integration). GNOME uses GTK library for GUI. GLib - objects for C; lib Pango - visualization of text; lib ATK - accessibility support for handicaped.

GDM

KDE

KDE uses Qt (C++ library). DCOP - system for communication among applications + user scripting. See kdcop. Ex: 

dcop kdesktop KBackgroundIface changeWallpaper ...
^ client ^ object ^ command ^ parameters

Private Network - connect two PCs via ehternet

Let's call the first computer "server" and the other one "client". IPs 192.168.1.x are allocated to private networks, they can never appear on the internet; 192.168.1.0 is address of the network, you can use e.g. addresses 192.168.1.1 - 192.168.1.254. (Class C is 192.168.0.0 - 192.168.255.255)


Linux & Media (audio, video)

Video

A bit of theory

I shall present here some basic terms related to video. My explanation is quite naive as I don't know much about it myself so sorry for inaccuracies.Read more e.g. in the documentation of MPlayer.

Video file format (container)
How the video and audio streams (+ meta information) are packed together. Different file formats support different combinations of video and audio codecs. Examples of video file formats are avi, mpeg4 ps (used with dvd), QuickTime (mov, mp4, mpg), ogg/ogm (Ogg Media File). Avi can include for example mp3 audio and mpeg1/2 video.
Video format/codec
How the video is compressed. mpeg1/2, divX, wmv, ogg theora.
Audio format/codec
How the audio is compressed. mp3, wma, ac3. ogg vorbis.
VBR/CBR/ABR (Variable Bit Rate/Constant? B.R./Average B.R.)
Different types of audio bitstreams, used e.g. in MP3.
Codec (COmpressor-DECompressor)
Wikipedia: codec is a "... program capable of performing transformations on a data stream ...". Most codecs are lossy - the quality decreases.

Software

You can grab video from a stream e.g. with the VLC media player. The resulting file is likely to be quite huge. You can compress it by encoding it - use e.g. avidemux. It's quite easy to encode with Avidemux (v. 2.0.38):

  1. Select an output format (e.g. avi), video codec (e.g. Lav Mpeg4) and audio codec (e.g. Lame = MP3). Then you can configure the codecs:
    1. Video - select one of the possible ways: 1 pass bitrate (you must set the desired bitrate); 1 pass quantization (you must set the quantization factor Q - between 1, 32, 1 is the highest quality & biggest; 4 or 5 should be quite OK; 2 pass (a good bitrate is determined automaticaly); ...).
    2. Audio - select the rate (the default 128 kbits represents quite high quality, I guess).
  2. Edit the file in any desired way -
  3. Enable the encoding - press the button "V Process[F5]" for video and "A Process[F6]" for audio. Without that they would just be copied without encoding.
  4. Save the file - the encoding will start. It may take quite a while, especially in the 2 pass mode.


 

Links