Difference between revisions of "Service/mirrors"

From Sugar Labs
Jump to: navigation, search
m (fix command)
m (move install instructions to the official mirror brain documentation)
Line 16: Line 16:
  
 
===Installation===
 
===Installation===
The following Recipe for installing mirrorbrain on ubuntu 9.04 is based on information at http://mirrorbrain.org/docs/ .
+
Please see http://mirrorbrain.org/docs/installation/debian/ .  Our Sugar Labs installation instructions have become the official mirrorbrain on ubuntu instructions.
  
====Install LAMP Server====
 
Install a standard Ubuntu LAMP server
 
 
====Download mirrorbrain====
 
<code>
 
wget http://mirrorbrain.org/files/releases/mirrorbrain-2.10.0.tar.gz
 
tar xzf mirrorbrain-2.10.0.tar.gz
 
</code>
 
 
====Install python dependencies====
 
<code>
 
sudo apt-get install python-sqlobject python-psycopg2
 
</code>
 
 
The python cmdln module is not prepackaged for Ubutnu so it must be installed manually.
 
 
<code>
 
wget http://cmdln.googlecode.com/files/cmdln-1.1.2.zip
 
unzip cmdln-1.1.2.zip
 
cd cmdln-1.1.2
 
sudo python setup.py install
 
</code>
 
 
====Install perl dependencies for scanner====
 
The mirrorbrain scanner is written in perl and requires several dependencies.
 
 
<code>
 
sudo apt-get install libconfig-inifiles-perl libwww-perl libdbd-pg-perl libdatetime-perl libdigest-md4-perl
 
</code>
 
 
====Build, Install, and Configure Apache2 modules====
 
Mirrorbrain requires several Apache modules, several of which must be built manually. Apache modules are built and installed using '''apxs2'''. (APache eXtenSion tool)  Apxs2 is in the apache2-threaded-dev package.
 
 
<code>
 
sudo apt-get install apache2-threaded-dev
 
</code>
 
 
=====Install and Configure mod_geoip=====
 
Mod_geoip is available as a prebuilt package.
 
 
<code>
 
sudo apt-get install libapache2-mod-geoip libgeoip-dev
 
</code>
 
 
Mod_geoip must be configured to find the the GeoIP data set.
 
 
<code>
 
sudo sh -c "cat > /etc/apache2/mods-available/geoip.conf << EOF
 
<IfModule mod_geoip.c>
 
  GeoIPEnable On
 
  GeoIPOutput Env
 
  GeoIPDBFile /var/lib/GeoIP/GeoIP.dat Standard
 
</IfModule>
 
EOF
 
"
 
</code>
 
 
Download GeoIP data set
 
<code>
 
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
 
sudo apt-get install gzip
 
gunzip GeoIP.dat.gz
 
sudo mkdir /var/lib/GeoIP
 
sudo cp GeoIP.dat /var/lib/GeoIP/GeoIP.dat
 
</code>
 
 
Enable module.
 
<code>
 
sudo a2enmod geoip
 
</code>
 
 
Restart Apache
 
<code>
 
sudo /etc/init.d/apache2 restart
 
</code>
 
 
=====Install and Configure mod_form=====
 
Mod_from must be build from sctatch.
 
 
FIXME correct directory
 
Download source
 
<code>
 
wget http://apache.webthing.com/svn/apache/forms/mod_form.c
 
wget http://apache.webthing.com/svn/apache/forms/mod_form.h
 
</code>
 
 
build mod_form
 
<code>
 
sudo apxs2 -cia mod_form.c
 
</code>
 
 
create loader
 
<code>
 
sudo sh -c "cat > /etc/apache2/mods-available/form.load << EOF
 
LoadModule form_module /usr/lib/apache2/modules/mod_form.so
 
EOF
 
"
 
</code>
 
 
Enable module.
 
<code>
 
sudo a2enmod form
 
</code>
 
 
Restart Apache
 
<code>
 
sudo /etc/init.d/apache2 restart
 
</code>
 
 
=====Configure mod_dsd=====
 
configure mod_dsd
 
<code>
 
sudo sh -c "cat > /etc/apache2/mods-available/dbd.conf << EOF
 
  <IfModule mod_dbd.c>
 
    DBDriver pgsql
 
    # note that the connection string (which is passed straight through to
 
    # PGconnectdb in this case) looks slightly different - pass vs. password
 
    DBDParams 'host=localhost user=mb password=12345 dbname=mb_sugar connect_timeout=15'
 
  </IfModule>
 
EOF
 
"
 
</code>
 
 
Enable module.
 
<code>
 
sudo a2enmod dbd
 
</code>
 
 
Restart Apache
 
<code>
 
sudo /etc/init.d/apache2 restart
 
</code>
 
 
=====Install and configure mod_mirrorbrain=====
 
Build mod_mirrorbrain
 
<code>
 
sudo apxs2 -cia mod_mirrorbrain.c
 
</code>
 
 
create module loader
 
<code>
 
sudo sh -c "cat > /etc/apache2/mods-available/mirrorbrain.load << EOF
 
LoadModule mirrorbrain_module /usr/lib/apache2/modules/mod_mirrorbrain.so
 
EOF
 
"
 
</code>
 
 
Enable module.
 
<code>
 
sudo a2enmod mirrorbrain
 
</code>
 
 
Restart Apache
 
<code>
 
sudo /etc/init.d/apache2 restart
 
</code>
 
 
====Build and Install helper programs====
 
build and install geoiplookup
 
<code>
 
gcc -Wall -lGeoIP -o geoiplookup_continent geoiplookup_continent.c
 
sudo cp geoiplookup_continent /usr/bin/geoiplookup_continent
 
</code>
 
 
Install scanner
 
<code>
 
sudo cp ../tools/scanner.pl /usr/bin/scanner
 
</code>
 
 
====Install postgres====
 
<code>
 
sudo apt-get install postgresql-8.3
 
</code>
 
 
=====Create the postgresql user account and database=====
 
Switch to user postgress
 
<code>
 
sudo su - postgres
 
</code>
 
 
Create user
 
<code>
 
createuser -P mb
 
Enter password for new role:
 
Enter it again:
 
Shall the new role be a superuser? (y/n) n
 
Shall the new role be allowed to create databases? (y/n) n
 
Shall the new role be allowed to create more new roles? (y/n) n
 
</code>
 
 
Create database
 
<code>
 
createdb -O mb mb_sugar
 
createlang plpgsql mb_sugar
 
</code>
 
 
Exit user postrges
 
<code>
 
exit
 
</code>
 
 
=====Edit host-based authentication=====
 
add line 'local mb_sugar mb trust' to the end of pg_hba.conf
 
 
FIXME should not be trust on production machine
 
<code>
 
sudo vim /etc/postgresql/8.3/main/pg_hba.conf
 
</code>
 
 
Start the potgres server
 
<code>
 
sudo /etc/init.d/postgresql-8.3 restart
 
</code>
 
 
=====Import intial mirrorbrain Data=====
 
Import table structure, and initial data
 
<code>
 
psql -U mb -f sql/schema-postgresql.sql mb_sugar
 
psql -U mb -f sql/initialdata-postgresql.sql mb_sugar
 
</code>
 
 
=====Create mirrorbrain user and group=====
 
<code>
 
sudo groupadd -r mirrorbrain
 
sudo useradd -r -g mirrorbrain -s /bin/bash -c "MirrorBrain user" -d /home/mirrorbrain mirrorbrain
 
</code>
 
 
=====Create mirrorbrain.conf=====
 
<code>
 
sudo sh -c "cat > /etc/mirrorbrain.conf << EOF
 
[general]
 
instances = samba
 
[samba]
 
dbuser = mb
 
dbpass = 12345
 
dbdriver = postgresql
 
dbhost = 127.0.0.1
 
# optional: dbport = ...
 
dbname = mb_sugar
 
[mirrorprobe]
 
mailto = dfarning@sugarlabs.org
 
EOF
 
"
 
</code>
 
 
Set permission and priviglies
 
<code>
 
sudo chmod 0604 /etc/mirrorbrain.conf
 
sudo chown root:mirrorbrain /etc/mirrorbrain.conf
 
</code>
 
 
=====Test mirrorbrain=====
 
<code>
 
./mirrordoctor.py
 
</code>
 
 
====Create VirtualHost====
 
<code>
 
sudo sh -c "cat > /etc/apache2/sites-available/mirrorbrain << EOF
 
  <VirtualHost 127.0.0.1>
 
    ServerName mirrors.sugarlabs.org
 
    ServerAdmin webmaster@mirrorbrain.org
 
    DocumentRoot /var/www/downloads
 
    ErrorLog    /var/log/apache2/mirrors.sugarlabs.org/error_log
 
    CustomLog    /var/log/apache2/mirrors.sugarlabs.org/access_log combined
 
    <Directory /var/www/downloads>
 
      MirrorBrainEngine On
 
      MirrorBrainDebug Off
 
      FormGET On
 
      MirrorBrainHandleHEADRequestLocally Off
 
      MirrorBrainMinSize 2048
 
      MirrorBrainHandleDirectoryIndexLocally On
 
      MirrorBrainExcludeUserAgent rpm/4.4.2*
 
      MirrorBrainExcludeUserAgent *APT-HTTP*
 
      MirrorBrainExcludeMimeType application/pgp-keys
 
      Options FollowSymLinks Indexes
 
      AllowOverride None
 
      Order allow,deny
 
      Allow from all
 
    </Directory>
 
</VirtualHost>
 
EOF
 
"
 
</code>
 
 
Enable site.
 
<code>
 
sudo a2ensite mirrorbrain
 
</code>
 
 
Restart Apache
 
<code>
 
sudo /etc/init.d/apache2 restart
 
</code>
 
  
 
<noinclude>{{ GoogleTrans-en | es =show | bg =show | zh-CN =show | zh-TW =show | hr =show | cs =show | da =show | nl =show | fi =show | fr =show | de =show | el =show | hi =show | it =show | ja =show | ko =show | no =show | pl =show | pt =show | ro =show | ru =show | sv =show }}</noinclude>
 
<noinclude>{{ GoogleTrans-en | es =show | bg =show | zh-CN =show | zh-TW =show | hr =show | cs =show | da =show | nl =show | fi =show | fr =show | de =show | el =show | hi =show | it =show | ja =show | ko =show | no =show | pl =show | pt =show | ro =show | ru =show | sv =show }}</noinclude>

Revision as of 11:56, 30 September 2009

Team Home   ·   Join   ·   Contacts   ·   Resources   ·   FAQ   ·   Roadmap   ·   To Do   ·   Meetings

Introduction

A content delivery network or content distribution network (CDN) is a system of computers containing copies of data, placed at various points in a network so as to maximize bandwidth for access to the data from clients throughout the network. A client accesses a copy of the data near to the client, as opposed to all clients accessing the same central server, thereby causing a bottleneck near that server.

Mirrorbrain, Bouncer, Fedora Mirror Manager, and Cacheboy are four possible choices for scaling up the Sugar Labs content delivery net work. Long term, Cacheboy looks like it might be the best fit for Sugar Labs, but the project must become more stable before becoming Sugar Labs primary CDN. Bouncer is currently being used as successfully by Mozilla. But development ended at the end of 2008. Mozilla is currently investigating Mirrorbrain. Mirrorbrain looks promising.

Goals

  • Reduce bandwith at primary download server.
  • Improve quality of service for users.
  • Move content closer to users.

Mirrorbrain

Mirrrorbrain is used by several major project including opensuse and openoffice. It is quite stable, under activate development, and well documented,

Installation

Please see http://mirrorbrain.org/docs/installation/debian/ . Our Sugar Labs installation instructions have become the official mirrorbrain on ubuntu instructions.


Subpages