Jump to content

Machine/Discovery One/Node: Difference between revisions

From Sugar Labs
← Older edit
Dfarning (talk | contribs)
2,751 edits
m add standard db
Dfarning (talk | contribs)
2,751 edits
 
(9 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== ASLO Node ==
= ASLO Node =
While discussing activities.sugarlabs.org maintainability with Dogi this afternoon, he suggested that instead of thinking about web, proxy, and database nodes we create a standard ALSO node which we can adopt and tune via configs.
While discussing activities.sugarlabs.org maintainability with Dogi this afternoon, he suggested that instead of thinking about web, proxy, and database nodes we create a standard ALSO node which we can adopt and tune via configs.


== Components ==
= Components =


== Setup Network ==
Set the IP address for the each nic


=== Web ===
/etc/networks/interfaces
<pre>
# The loopback network interface
auto lo
iface lo inet loopback
 
# The primary network interface
auto eth0
iface eth0 inet static
        address 10.0.0.101
        netmask 255.255.255.0
        network 10.0.0.0
        broadcast 10.0.0.255
        gateway 10.0.0.1
 
#auto eth1
iface eth1 inet static
        address 10.0.0.201
        netmask 255.255.255.0
        network 10.0.0.0
        broadcast 10.0.0.255
        gateway 10.0.0.1
</pre>
 
restart the network
 
/etc/init.d/networking restart
 
== Install php server ==


=== Database ===
===Install needed packages===
===Install needed packages===
  apt-get install mysql-client-5.0 mysql-server-5.0
  sudo apt-get install gettext
 
===Install required php modules===
sudo pear config-set preferred_state beta
sudo pear install Archive_Zip
 
===Create user to run scripts===
sudo useradd -d /srv/activities activities
 
Make sure that all files in /srv/activities
* has apache(web-data) user
* ''activities'' group
* group write access
* umask is 002


===Create database and set permissions===
===Switch to activities user===
mysqladmin -u root -p create activities
mysql -u root -p


  GRANT ALL ON *.* TO 'activities'@'xxx.xxx.xxx' IDENTIFIED BY 'XXXX';
  sudo -i -u activities
exit;


===Configuring database===
===Checkout ASLO sources===
/etc/mysql/my.cnf


<pre>
git clone git://git.sugarlabs.org/slo-activities/mainline.git ~
[mysqld_safe]
git checkout production -b production
socket          = /var/run/mysqld/mysqld.sock
nice            = 0


[mysqld]
===Fetch external dependencies===
# * IMPORTANT
#  If you make changes to these settings and your system uses apparmor, you may
#  also need to also adjust /etc/apparmor.d/usr.sbin.mysqld.


user            = mysql
cd ~/site/vendors/
pid-file        = /var/run/mysqld/mysqld.pid
svn co http://svn.mozilla.org/libs/product-details
socket          = /var/run/mysqld/mysqld.sock
svn co http://svn.mozilla.org/libs/fizzypop.mozdev fizzypop
port            = 3306
svn co http://svn.mozilla.org/libs/phorms
basedir        = /usr
svn co http://svn.mozilla.org/libs/zxtm-api
datadir        = /var/lib/mysql
tmpdir          = /tmp
skip-external-locking




# Instead of skip-networking the default is now to listen only on
===Set up NFS mount points===
# localhost which is more compatible and is not less secure.
Create directories
# bind-address            = 127.0.0.1
mkdir -p ~/files
mkdir -p ~/staging


# * Tuning
Add the following lines to the end of `/etc/fstab'
#


tmp_table_size          = 16M
  sunjammer.sugarlabs.org:/srv/nfs/activities/files /srv/activities/files nfs rsize=8192,wsize=8192,timeo=14,intr
  sunjammer.sugarlabs.org:/srv/nfs/activities/staging /srv/activities/staging nfs rsize=8192,wsize=8192,timeo=14,intr
innodb_buffer_pool_size = 1024M
key_buffer              = 64M
   
max_allowed_packet      = 16M
thread_stack            = 128K
thread_cache_size      = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover          = BACKUP
max_connections        = 151
   
table_cache            = 128
#thread_concurrency    = 10
query_cache_limit      = 2M
query_cache_size        = 32M


# * Logging and Replication
===First-time ASLO setup===
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
#log            = /var/log/mysql/mysql.log
#
# Error logging goes to syslog. This is a Debian improvement :)
#
# Here you can see queries with especially long duration
log_slow_queries        = /var/log/mysql/mysql-slow.log
long_query_time = 1
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#      other settings you may need to change.
#server-id              = 1
#log_bin                        = /var/log/mysql/mysql-bin.log
expire_logs_days        = 10
max_binlog_size        = 100M
#binlog_do_db          = include_database_name
#binlog_ignore_db      = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
# You might want to disable InnoDB to shrink the mysqld process by circa 100MB.
#skip-innodb
#
# * Federated
#
# The FEDERATED storage engine is disabled since 5.0.67 by default in the .cnf files
# shipped with MySQL distributions (my-huge.cnf, my-medium.cnf, and so forth).
#


#
Using ''~/aslo/config.php'' as template, setup ASLO settings in file ''~/site/app/config/config.php''. Template file was configured for ''dev'' environment, switch it to production by  setting ''SITE_STATE'' variable to ''production''.
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem


[mysqldump]
Setup other env defines in file ''~/site/app/config/config-local.php''
quick
define('DEBUG', 0);
quote-names
define('DEV', false);
max_allowed_packet      = 16M
define('QUERY_CACHE', true);


[mysql]
Create directories
#no-auto-rehash # faster start of mysql but no tab completition
mkdir -p ~/site/app/tmp/cache/persistent
mkdir -p ~/site/app/tmp/cache/models
mkdir -p ~/site/app/tmp/cache/views
mkdir -p ~/files/temp
mkdir -p ~/files/extracted
mkdir -p ~/data


[isamchk]
Minify css/js stuff
key_buffer              = 16M
~/aslo/minify.py


#
Merge ASLO gettext strings
# * NDB Cluster
~/aslo/po-compile.sh
#
# See /usr/share/doc/mysql-server-*/README.Debian for more information.
#
# The following configuration is read by the NDB Data Nodes (ndbd processes)
# not from the NDB Management Nodes (ndb_mgmd processes).
#
# [MYSQL_CLUSTER]
# ndb-connectstring=127.0.0.1


===Crontab for activities user===


#
@hourly $HOME/aslo/cron/hourly
# * IMPORTANT: Additional settings that can override those from this file!
# do not use @daily to make sure that we start job after aslo-proxy
#  The files must end with '.cnf', otherwise they'll be ignored.
01 08 * * * $HOME/aslo/cron/daily
#
!includedir /etc/mysql/conf.d/
</pre>


===Log Rotate===
===Create vhost===
/etc/logrotate.d/mysql-server
vim /etc/apache2/sites-available/activities.sl.org
<pre>
<pre>
# - I put everything in one block and added sharedscripts, so that mysql gets 
<VirtualHost *:80 *:443>
#  flush-logs'd only once.  
        ServerName activities.sugarlabs.org
#   Else the binary logs would automatically increase by n times every day.  
        ServerAlias activities2.sugarlabs.org # for test/install
# - The error log is obsolete, messages go to syslog now.
        ServerAdmin webmaster@sugarlabs.org
/var/log/mysql.log /var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log {
 
        daily
        DocumentRoot /srv/activities/site/app/webroot
        rotate 7
 
        missingok
        LogFormat "%h %V %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" \"%{HTTP_COOKIE}i\"" addons
        create 640 mysql adm
        CustomLog /srv/activities/data/access addons
        compress
        CustomLog /var/log/apache2/all-access.log vhost_combined
        sharedscripts
         #CustomLog /var/log/apache2/performance.log performance
         postrotate
</VirtualHost>
                test -x /usr/bin/mysqladmin || exit 0
                # If this fails, check debian.conf! 
                MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf"
                if [ -z "`$MYADMIN ping 2>/dev/null`" ]; then
                  # Really no mysqld or rather a missing debian-sys-maint user?
                  # If this occurs and is not a error please report a bug.
                  if ps cax | grep -q mysqld; then
                    exit 1
                  fi 
                else
                  $MYADMIN flush-logs
                fi
        endscript
}
</pre>
</pre>

Latest revision as of 11:24, 1 March 2010

ASLO Node

While discussing activities.sugarlabs.org maintainability with Dogi this afternoon, he suggested that instead of thinking about web, proxy, and database nodes we create a standard ALSO node which we can adopt and tune via configs.

Components

Setup Network

Set the IP address for the each nic

/etc/networks/interfaces 

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 10.0.0.101
        netmask 255.255.255.0
        network 10.0.0.0
        broadcast 10.0.0.255
        gateway 10.0.0.1

#auto eth1
iface eth1 inet static
        address 10.0.0.201
        netmask 255.255.255.0
        network 10.0.0.0
        broadcast 10.0.0.255
        gateway 10.0.0.1

restart the network 

/etc/init.d/networking restart

Install php server

Install needed packages

sudo apt-get install gettext

Install required php modules

sudo pear config-set preferred_state beta 
sudo pear install Archive_Zip

Create user to run scripts

sudo useradd -d /srv/activities activities

Make sure that all files in /srv/activities

  • has apache(web-data) user
  • activities group
  • group write access
  • umask is 002

Switch to activities user

sudo -i -u activities

Checkout ASLO sources

git clone git://git.sugarlabs.org/slo-activities/mainline.git ~
git checkout production -b production

Fetch external dependencies

cd ~/site/vendors/
svn co http://svn.mozilla.org/libs/product-details
svn co http://svn.mozilla.org/libs/fizzypop.mozdev fizzypop
svn co http://svn.mozilla.org/libs/phorms
svn co http://svn.mozilla.org/libs/zxtm-api


Set up NFS mount points

Create directories 

mkdir -p ~/files
mkdir -p ~/staging

Add the following lines to the end of `/etc/fstab' 

sunjammer.sugarlabs.org:/srv/nfs/activities/files /srv/activities/files nfs rsize=8192,wsize=8192,timeo=14,intr
sunjammer.sugarlabs.org:/srv/nfs/activities/staging /srv/activities/staging nfs rsize=8192,wsize=8192,timeo=14,intr

First-time ASLO setup

Using ~/aslo/config.php as template, setup ASLO settings in file ~/site/app/config/config.php. Template file was configured for dev environment, switch it to production by setting SITE_STATE variable to production.

Setup other env defines in file ~/site/app/config/config-local.php 

define('DEBUG', 0);
define('DEV', false);
define('QUERY_CACHE', true);

Create directories 

mkdir -p ~/site/app/tmp/cache/persistent
mkdir -p ~/site/app/tmp/cache/models
mkdir -p ~/site/app/tmp/cache/views
mkdir -p ~/files/temp
mkdir -p ~/files/extracted
mkdir -p ~/data

Minify css/js stuff 

~/aslo/minify.py

Merge ASLO gettext strings 

~/aslo/po-compile.sh

Crontab for activities user

@hourly $HOME/aslo/cron/hourly
# do not use @daily to make sure that we start job after aslo-proxy
01 08 * * * $HOME/aslo/cron/daily

Create vhost

vim /etc/apache2/sites-available/activities.sl.org 

<VirtualHost *:80 *:443>
        ServerName activities.sugarlabs.org
        ServerAlias activities2.sugarlabs.org # for test/install
        ServerAdmin webmaster@sugarlabs.org

        DocumentRoot /srv/activities/site/app/webroot

        LogFormat "%h %V %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" \"%{HTTP_COOKIE}i\"" addons
        CustomLog /srv/activities/data/access addons
        CustomLog /var/log/apache2/all-access.log vhost_combined
        #CustomLog /var/log/apache2/performance.log performance
</VirtualHost>
Retrieved from "https://wiki.sugarlabs.org/index.php?title=Machine/Discovery_One/Node&oldid=49074"