torstai 25. huhtikuuta 2013


Fabric is a Python (2.5 or higher) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.

It provides a basic suite of operations for executing local or remote shell commands (normally or via sudo) and uploading/downloading files, as well as auxiliary functionality such as prompting the running user for input, or aborting execution.

More from Fabric documentation -- >

$ sudo apt-get install fabric

Create new python file

$ nano

insert code

def hi():
print("Hello World!")

$ cat

$ fab -l (list functions)

Install ssh + server
$ apt-get install openssh-server

import fabric library to

from fabric.api import*

env.hosts=["localhost", ""]

passwd on live cd

Run function from python script

$ fab koneenimi

Make ssh login automatic with keys

$ ssh-keygen
$ ssh-copy-id xubuntu@localhost

sudo visudo

add line under #allow members of group sudo to execute any command


to see if package installed correctly
$ dpkg -s softwarename

And now I just post wall of code here. Remember that this is Python so indents must be correct to get this to work!

from fabric.api import*

env.hosts=["localhost",""] #, "simo@"
env.roledefs={"server":["localhost"], "workstation": ["", ""]}


def hi():
    print("Hello World!")

@roles ("server")
def koneenimi():
def moi():
    run("echo moi")

@with_settings (warn_only=True)
@ hosts("localhost")
def komento(cmd):
    with settings(warn_only=True):
        if run(cmd).failed:

def skomento(cmd):

def asennus():
    sudo("apt-get update")
    sudo("apt-get -y install e3")

def puppetasennus():
    sudo("apt-get update")
    sudo("apt-get -y install puppet")

def puppetmoduli():
    put("/etc/puppet/modules/apache2", "/etc/puppet/modules/", use_sudo=True)

def asen(paketti):
    sudo("apt-get update")
    sudo("apt-get -y install %s" % paketti)

def paikallinenkomento():
    local("echo asensin e3 paketin %s' >> loki.log" % env.hosts)
    sudo("apt-get -y install e3")

def send():
    put("/home/xubuntu/gittiprojekti/loki.log", "/tmp/loki.log",use_sudo=True)

def lataus():
    get("/tmp/loki.log", "/home/xubuntu/gittiprojekti/loki.log" + "."
    run("echo jee")

def b():
    put("loki.log.localhost", "/tmp/loki.log.localhost", mode=0755)
    sudo("chown root.root /tmp/loki.log.localhost")

def a():
        put("loki.log.localhost", "/tmp/loki.log.localhost", mirror_local_mode=True)

def testi():
    with settings(user="niki", host_string="localhost"):
    with settings(user="pena", host_string=""):

def cdtesti():
    with cd("/tmp/"):

def test():

def production():

Remove CD-ROM from apt-get repositories

To remove the cd-rom from apt-get repositories:

The fix is easy. We need to remove the CD-ROM as a source for apt-get.
To do this, edit the following file: /etc/apt/sources.list

Comment or delete the line for the CD-ROM, save the changes, and try apt-get again.
This time, it should use the online repositories.

perjantai 19. huhtikuuta 2013

Git homework

Home assignment was to learn GIT.
a) create one person local git storage
b) do multiple commits, some very random, and then return to the last working set of files (git reset)
c) try some git user interface for example giggle
d) put your git on server
e) clone git files for multiple computers and try that files are synchronizing between the computers

I am using Oracle VM Virtualbox with Xubuntu 12.10 64bit

Update repositories first
$ sudo apt-get update
Install git
$ sudo apt-get install git
Create new project folder somewhere, for example
$ mkdir project
$ cd project
Make this project to git project with
$ git init
(This creates a new subdirectory named .git that contains all of your necessary repository files — a Git repository skeleton. At this point, nothing in your project is tracked yet.)

Then create your code, text or what ever document.
$ nano
Make the file ready for git
$ git add . (dot includes all files in that folder)
Commit it to git
$ git commit
Add some commit message and that's it.

$ git log
to see log of committed files.

The main tool you use to determine which files are in which state is the git status command. If you run this command directly after a clone, you should see something like this:

$ git status
# On branch master
nothing to commit (working directory clean)

Resetting git 

One of the common undos takes place when you commit too early and possibly forget to add some files, or you mess up your commit message. If you want to try that commit again, you can run commit with the

$ git commit --amend

git reflog is useful command to see where the project head is. You can think head as a metro train and the head means the first train. All other trains are previous commits in order.

$ git reflog

-h command shows good information, for example

$ git reset -h

And if you want to go back (reset the head) use
$ git reset --hard (commit number here)
$ git reset --hard 5bca300

BUT, if you want to push this now on the server you cant. You have to force it with
$ git push --force


$ sudo apt-get install giggle

Giggle is graphical user interface for Git. In the screenshot you can see that my project folder has some random files and giggle shows graphical line for all the commits I have done. There is a short log, authors name and date of course.

From offline to online --> tutorial

I created a git repository on my raspberryPi.

$ ssh myname@ipofmyserver
server$ mkdir gitvarasto; cd gitvarasto/
server$ git init --bare --shared
$ exit

Then I made my local copy sync with the server

$ git remote add origin ssh://myname@ipofmyserver/home/myname/gitvarasto
$ git push origin master 
$ git branch --set-upstream master origin/master 
$ git pull && git push

As we can see my virtualbox computer's Git is set to use ssh to connect raspberryPi

Now just work normally with the files 

$ git pull && git push  
edit some files
$ git add . && git commit
$ git pull && git push

Then I opened my laptop, made a new folder and cloned the files from server

$ git clone ssh://myname@ipofmyserver/home/myname/gitvarasto

It copied all the files I have been working on other computer, nice. I also tried to edit a file on my laptop, then committed it and from virtualbox: $ git pull && git push and it synchronized.

And from giggle we can see nice graphical history what has happened. Who has modified the files and short log.

keskiviikko 17. huhtikuuta 2013

Git - version control system (notes from class)

"Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency."

To start a Git project create project folder somewhere, for example /home/xubuntu/project

$ cd /home/xubuntu
$ mkdir project
$ cd project

Make a hidden .git folder inside project folder

$ git init

with  status command we can see the status of course.

$ git status

Create a new file, for example:

$ nano teksti.txt

Add some text, save and quit the file.

Then add it to Git and commit.

$ git add . (remember the dot here it copies all files then.)
$ git commit

Add some commit message and apply. With git log command you can see what happened.

$ git log

Log command shows for example file author, time and the message what has changed.

If something goes wrong you can reset everything, but  be very careful with this command.

$ git reset --hard HEAD

Now is time to register in Git. 

You can pull all codes from folder with:

$ git clone git:// /home/xubuntu/mygrit

Where "shnigi" is your account name "try_git" your project folder or name and after that the path where to save the file.

From offline to online -->

$ sudo apt-get install giggle (gui for git ?)

lauantai 13. huhtikuuta 2013

/etc/skel - public_html directory for new user and test

I wanted to test how the /etc/skel folder works so I created a new folder in the skel folder named public_html.

$ cd /etc/skel
$ sudo mkdir public_html

Then I created new user with

$ sudo adduser testuser

I noticed the public_html folder was copied from the /etc/skel to the testuser's home folder. /home/testuser/public_html I copied the basic HTML file there edited it a little bit and then pointed my browser to http://localhost/~testuser and noticed it works !

User homepages on Apache + php + password protection

I didn't test if the user homepages are enabled by default but setting it on makes this happen: anything you put in public_html in your home directory is published on the web.

Turn the mod on and restart service:

$ sudo a2enmod userdir
$ sudo service apache2 restart

create a folder named "public_html" on your home directory and put "index.html" there with some HTML code.

Check your name if you don't know it:

$ whoami

Then surf to http://localhost/~yourname

To allow php for users.

$ sudo apt-get install libapache2-mod-php5
$ cd /etc/apache2/mods-enabled/
$ sudoedit php5.conf 
Comment the lines out from FilesMatch 
This is simple php file to test if it is working
<title>This website supports php!</title>
print "this is php, 1+2 is ".(1+2);

$ sudo service apache2 restart

 Password protected site.

Again create a new folder for your site which you want to have password protection.

$ mkdir password
$ htpasswd -c .htpasswd xubuntu
-c = create. makes a new file, if a file exists it is overwritten. This creates hidden file .htpasswd for user xubuntu.

$ sudoedit /etc/apache2/sites-available/default

Edit apache default settings by adding this:

   <Directory /home/xubuntu/public_html/password/>
        AllowOverride AuthConfig
        AuthName "Please insert your login data."
        AuthType Basic
        AuthUserFile /home/xubuntu/public_html/password/.htpasswd
        AuthGroupFile /dev/null
        require user xubuntu

$ sudo service apache2 restart

And for me this also worked. Original instructions:


It is better to create .htaccess file under the website folder and add:

AuthUserFile /home/location/.htpasswd
AuthType Basic
AuthName "Login"
Require valid-user

Puppet, package-file-service

I'm still bit confused how puppet works but still I got it to work somehow. My home assignment was to install two demons with package-file-service technique, which means that Puppet first checks that the package is installed, then service turns it on and lastly copies demon settings from template located somewhere and after that "kicks" the demon that settings has been changed.

I had already done the apache2 so I mainly followed the script written in the class. I started by creating folders for sshd module, located in /etc/puppet/modules/

First I created main module folder named sshd and then two directories under sshd folder:

$ mkdir sshd; cd sshd
$ mkdir manifests/ templates/

Then I updated repositories and installed openssh-server

$ sudo apt-get update && sudo apt-get -y install openssh-server

After that I copied the sshd_config file to sshd/templates/sshd_config.erb

$ cp /etc/ssh/sshd_config templates/

Rename copied file to .erb and go to the manifests folder. There create new manifest
$ nano init.pp

Text to init.pp:

class sshd {

package { 'openssh-server':
ensure => installed,

service {'ssh':
ensure => running,
require => Package["openssh-server"],

file {'/etc/ssh/sshd_config':
ensure => file,
content => template ("/etc/puppet/modules/sshd/templates/sshd_config.erb"),
require => Package["openssh-server"],
notify => Service["ssh"],

# In this case the template path should be sshd/sshd_config.erb

Notify that my template file is in /etc/puppet/modules/sshd/templates/sshd_config.erb
And to run the module:

$ sudo puppet apply -e "include sshd"

This seems to work.

Then the apache2 module:

Same folder hierarchy for apache2, under main folder manifests folder and templates if wanted.

In the init.pp

class apache2 {

package { 'apache2':
ensure => installed,

service {'apache2':
ensure => running,
require => Package["apache2"],

file { '/var/www/testsite':
ensure => directory,
mode => 644,
require => Package["apache2"],
notify => Service["apache2"],

I had already installed apache2 so I turned it off

$ sudo service apache2 stop

And run my apache2 module:

$ sudo puppet apply -e "include apache2"

As we can see apache2 is not running in the center and after I run the module apache2 is running.

And now to combine the two modules together:

$ sudo puppet apply -e "include apache2, sshd"

Test scenario if this is working:

$ sudo service apache2 stop
$ sudo service ssh top

And if we want to see what happened:

$ sudo service apache2 status / ssh status

And after the puppet run we can see it changed both services to run

Worked. And to see how the directory hierarchy is set I installed "tree"

$ sudo apt-get install tree

headed to modules folder and

$ tree

This post was home assignment for Puppet course:

perjantai 12. huhtikuuta 2013

Webmin for RaspberryPi

I wanted to try webmin which is a program or more likely GUI that can monitor your system from the browser. Webadmin is a good tool for server admin if you don't remember all the Linux codes. I found the instructions from the internet and again I am just going to copy paste it here as it is very simple process and worked out of the box.

choose a place where you want to install it and:

$ mkdir webmin
$ cd webmin
$ wget
$ gunzip webmin-1.580.tar.gz
$ tar xf webmin-1.580.tar
$ cd webmin-1.580
$ sudo ./ /usr/local/webmin

Then set settings as you want or if you want to use the defaults press enter about 3-7 times.
After this it should say installation completed and you can find the webadmin page by pointing your browser for example http//localhost:10000 (10000 is the default port)

Next I have to open ports from my router to access my Pi outside my home network. 

torstai 11. huhtikuuta 2013

Puppet - file, package, service

Scenario: I want to install ssh server, replace the default config with my own settings and check that the service is running.

First create a new module folder named for example sshd and inside it folder manifests. Then create init.pp file inside manifests with
$ sudo nano init.pp
inside init.pp write this:

class sshd {
package { 'openssh-server':
ensure => "installed",

file { '/etc/ssh/sshd_config':
content => template("/home/xubuntu/Templates/sshd_template.erb"),
require => Package["openssh-server"],
notify => Service["ssh"],

service { 'ssh':
ensure => 'running',
enable => 'true',
require => Package["openssh-server"],

hasstatus => 'false',
status => "/etc/init.d/ssh status|grep running",

Then create new template file in /home/xubuntu/Templates/sshd_template.erb inside the .erb file put your own sshd config file.

Run puppet with: $ sudo puppet apply -e "include sshd"

keskiviikko 10. huhtikuuta 2013

Install google chrome on linux


sudo apt-get install libgconf2-4

32bit installs:
wget -P /tmp
sudo dpkg -i /tmp/libudev0_175-0ubuntu19_i386.deb
rm /tmp//libudev0_175-0ubuntu19_i386.deb

64bit installs:
wget -P /tmp
sudo dpkg -i /tmp/libudev0_175-0ubuntu13_amd64.deb
rm /tmp/libudev0_175-0ubuntu13_amd64.deb

Original tutorial:

To get started, press Ctrl – Alt – T to open Terminal. When it opens, run the command below to add Linux-repository public key from Google..

$ wget -q -O - | sudo apt-key add -

 $ sudo gedit /etc/apt/sources.list.d/google.list

Then copy and paste the line below into the file and save.deb stable main

After saving the file, run the commands below to update your system.

$ sudo apt-get update

Finally, run the commands below to install Google Chrome Stable version.

$ sudo apt-get install google-chrome-stable

lauantai 6. huhtikuuta 2013

Puppet facter facts

If you want to see all the facts about your computer use show facter facts command
$ facter

and show all facter facts, and those defined for use in puppet (run as root)
$ sudo facter -p

List of Facter facts

Puppet modules and classes

Puppet Templates / Puppet Modules

Puppet modules can include multiple manifests, files and many things. Whole manifest is put inside class modulename {} and make a working folder hierarchy for it.

So I went to /etc/puppet/modules
$ cd /etc/puppet/modules
and created a new module named apache2
$ mkdir apache2; cd apache2; mkdir manifests
then I created new init.pp file
$ nano init.pp
And here is the text for my manifest it installs apache2 and makes sure it is runnig, then creates a new directory in /var/www/testsite and adds empty index.html file:

# this manifest installs apache2

class apache2 {

#Ensure that apache2 is installed and running
package { 'apache2':
ensure => installed,

service {'apache2':
ensure => running,

#Ensure there is directory for our site
file { '/var/www/testsite':
ensure => directory,
#Rear and write for me, read for others.
mode => 644,

#Ensure that we have index page in testsite
file {'/var/www/testsite/index.html':
ensure => file,

And to reveal it
$ sudo puppet apply -e "include apache2"
Which seems to work!

Now I can include the class from any manifest, without having to cut and paste anything.


Template is a file in somewhere which content is copied. Template file ending is .erb

I added this line under the ensure index.html is present:
content => template ("/home/shnigi/templeitti.erb"),

Then I created a new file named templeitti.erb in /home/shnigi/ and added some HTML code to it.

$ sudo puppet apply -e "include apache2"

Everything went fine in terminal, then I pointed my browser to http://localhost/testsite/ and noticed the html was copied from the erb file to the index.html file. I'm starting to understand this thing.

Puppet manifest basic examples

I wanted to understand puppet better so I did my first homework again. Here are some basic manifests and I will also explain those.

Create a folder for Puppet mount it and create file named init.pp.

$ mkdir puppet
$ mkdir manifests
$ cd pupppet/manifests
$ nano init.pp

Inside the init file we write:

file {'just testing puppet': 
path => '/tmp/this is just testfile',
ensure => present, 
mode => 0640, 
content => "contains test content", }

cd to manifests folder which we just created, cd puppet/manifests

$ puppet apply init.pp
notice: /Stage[main]//File[just testing]/ensure: created 
notice: Finished catalog run in 0.02 seconds

See your tmp folder and notice it created that file.

Second manifest. Lets install apache and make sure it is running. You can edit the init.pp or create a new file ending .pp

package { 'apache2': 
 ensure => installed } 
service { 'apache2':
 ensure => running }

Lets run that. Notice that you have to run it with root rights because you have to install new service.

$ sudo puppet apply init.pp 
warning: Could not retrieve fact fqdn 
notice: Finished catalog run in 0.39 seconds

For me it worked out of the box. I just read that someone had to apply it twice because it tries to run the service before installing it.

To see if apache2 is running use:
$ sudo service apache2 status
Apache2 is running (pid12758).

torstai 4. huhtikuuta 2013

Linux folder hierarchy

I am too lazy to write my own stuff because other posts on the internet explain this so well. Original post:

The /bin directory is where many of the basic Linux commands are housed. The /bin directory is in all standard user $PATHS, meaning that any executable contained within /bin can be executed from anywhere in the directory structure. The globalization of the /bin directory allows the user to run the ls (and other commands) on current working directories and files.

The /boot directory houses all the boot code for the Linux system. Crucial files are housed within this directory and, unless a kernel recompilation is necessary, it is best not to touch the contents of /boot.

The /dev directory seems to be the biggest mystery to new Linux users. To understand this directory, you need to keep in mind that everything is treated as a file in Linux—even devices such as printers, scanners, and hard drives.

Since everything is a file in Linux, every file needs a centralized location to be housed, and what better place to house device files than the /dev directory?

By running the ls command from within the /dev directory, you will see entries such as audio,cdrom, hda, sda, and ttyS0. Each of these entries represents a possible hardware (or system) device for the machine. For example, on my machine, I know that /dev/hda is the first primary partition on my hard drive.

The /etc directory is a vital part of the Linux system because it contains the majority of the necessary configuration files and resources. Most of these files are saved in text format, so they are editable with any text editor. The permissions of the /etc directory dictate that only the root user can modify the contents. Do not change this permission.

The /home directory houses all the individual user account information as well as the users' personal files. If a particular Linux system has normal user accounts such as Jack, John, and Jim, then /home will contain directories for each user (named after each respective user).

The /lib directory contains all the shared library files necessary to run the Linux system. These files are all dynamically linked from various places around the hierarchy.

The /lost+found directory contains stray files that have been picked up after a system crash. Typically, after a crash, lost files will appear as numbered files (not as the original filenames).

The /opt directory is simply a location where larger packages (such as StarOffice, OpenOffice, and Applixware) should be installed.

The /proc directory contains virtual files for various processes on a machine such as the CPU, RAM, I/O Ports, and Interrupts.

The /sbin directory contains many applications that are reserved for the root user. The /sbindirectory should house any applications that are necessary to start system processes, as well as many of the configuration utilities (such as netconf).

The /tmp directory holds the temporary files that are needed for various applications and processes.

The /usr directory contains a number of very important subdirectories such as:

This directory is like /bin, only it contains more advanced commands and many user-installed commands.
This is where the kernel source is located.
This directory is commonly used to install packages from source.
This directory contains system commands that can be executed only by the system or the root user.
This directory contains the documentation installed on the system.
This directory contains the files used by the man help system.
The /var directory contains system files, such as logs, and often-changing user/system files, such as print spoolers and mail spoolers.

Linux permissions

Original instructions:

While your computer will only have one keyboard and monitor, it can still be used by more than one user. That is why every file got its own permissions.

Permissions are divided in three sections "---" one "---" two "---" three. First one is for the user rights the "owner", second one is for group members and last one is for everybody else.

To see what permissions a file has type:
$ ls -l filename

-rw-rw-r-- 1 me me 1097374 Sep 26 18:48 filename

The file "some_file" is owned by user "me"
User "me" has the right to read and write this file
The file is owned by the group "me"
Members of the group "me" can also read and write this file
Everybody else can read this file

And to loan my studying material:


If you know how to calculate binaries or if you don't read my binary post below; It is easy to think of the permission settings as a series of bits (which is how the computer thinks about them). Here's how it works:

rwx rwx rwx = 111 111 111
rw- rw- rw- = 110 110 110
rwx --- --- = 111 000 000


rwx = 111 in binary = 7
rw- = 110 in binary = 6
r-x = 101 in binary = 5
r-- = 100 in binary = 4

Now, if you represent each of the three sets of permissions (owner, group, and other) as a single digit, you have a pretty convenient way of expressing the possible permissions settings. For example, if we wanted to set some_file to have read and write permission for the owner, but wanted to keep the file private from others, we would:

$ chmod 600 some_file

In the link provided on top of this post there you can find better tables how to use chmod with numbers. 


The group ownership of a file or directory may be changed with chgrp. This command is used like this:

$ chgrp new_group some_file

Puppet Template

Create a new file named templeitti.erb, write some text there "blah blah, blah" and save it. To puppet init.pp write:

file {'testfile':
      path    => '/tmp/testfile',
      ensure  => present,
   content => template("/home/xubuntu/templeitti.erb"),

This creates a new file named "testfile" under /tmp folder. The file gets its contents from the template file which is originally located in /home/xubuntu/templeitti.erb

Puppet templates documentation page: Puppet Templates Documentation


Understanding binary codes

Binary that weird looking number line. It is actually very easy to understand how to calculate binaries. Normal binary consists of numbers one "1" and zero "0" and the number 1 means that the value is turned ON and if the number is 0 it means it is turned OFF.

We start from number one always multiplying it by two. We get numbers 1,2,4,8,16,32,64,128. Eight numbers. Then we make a table:


Remember, 0 is OFF. 1 is ON. Now the binary is 0 because every number is turned off. Lets turn something on.


Now two values are turned on, and guess what; just simply calculate the numbers together. Number two is ON so we have 2 and the number 8 is turned ON so we have numbers 2 and 8. 2 + 8 = 10. The binary value is ten. Simple as that.


And if we turn everything on we have 1+2+4+8+16+32+64+128 = 255. 

Now you should know how to calculate binary numbers. You can easily change IP address to binary form for example or vice versa. 

keskiviikko 3. huhtikuuta 2013

Linux directories - The /etc/skel Directory

This post was so simple to understand that i'm going to copy it as it is. Original post:

The /etc/skel directory contains files and directories that are automatically copied over to a new user's home directory when such user is created by the useradd program.

A home directory, also called a login directory, is the directory on Unix-like operating systems that serves as the repository for a user's personal files, directories and programs, including personal configuration files. It is also the directory that a user is first in after logging into the system. The /etc directory and its subdirectories contain the many important configuration files for the system.

The useradd program is located in the /usr/sbin/ directory, and on most systems it is accessible only by the root (i.e., administrative) user. On some systems this program might be called adduser.

/etc/skel allows a system administrator to create a default home directory for all new users on a computer or network and thus to make certain that all users begin with the same settings or environment.

Several user configuration files are placed in /etc/skel by default when the operating system is installed. Typically they might include .bash_profile, .bashrc, .bash_logout, dircolors, .inputrc and .vimrc. The dots preceding the names of these files indicate that they are hidden files, i.e., files that are not normally visible in order to avoid visual clutter and help reduce the chances of accidental damage.

The contents of /etc/skel can be viewed by using the ls (i.e., list) command with its -a option (which shows all files and directories, including hidden ones), i.e.,

ls -a /etc/skel

The location of /etc/skel can be changed by editing the line that begins with SKEL= in the configuration file /etc/default/useradd. By default this line says SKEL=/etc/skel.

It is usually better to keep /etc/skel as small as possible and put system-wide configuration items into global configuration files such as /etc/profile. This is because the latter makes it much easier to update existing users' files because its settings take effect as soon as the system is turned on and apply to new users and old uses alike.

When a user is removed from the system by an administrator with the userdel command, that user's home directory, including the files and directories that have been copied into it from /etc/skel, remains intact.

The name of the directory skel is derived from the word skeleton, because the files it contains form the basic structure for users' home directories.

Testing the skel directory:

$ sudo su
# cd /etc/skel
# mkdir workdir
# touch script1 script2
# ls
script1 script2 workdir
# useradd -m -d /home/testdir test9
# ls /home/testdir
script1 script2 workdir