Platform Team/Server Kit/Guide/Usage Statistics

RRD (Round Robin Database) file data types
There is a standard set of rrd databases containing stats for journal, system, shell. For each activity there is an independent rrd database which tracks a standard set of parameters, the details for which are listed below.


 * database files conform Statistics identifiers list;
 * database file name for non-activities is in format .rrd;
 * database file name for activities is in format activity..rrd;
 * the set of database fields conform second level stat identifiers for particular stat.

Factors affecting network bandwidth
The only factor that really affects the network bandwidth usage is the resolution at which the data is being stored. Currently, the default step-size is 60 seconds.

Factors affecting database size on the client side
The factor that affects the database size on the client is the RRAs (Round Robin Archives) being cached on the Client XO.

The default RRA configuration on the client side reads something like

RRA:AVERAGE:0.5:1:4320 RRA:AVERAGE:0.5:5:2016

This means that there is one RRA (Round robin archive) within the DB that stores one value every minute for 4320 samples. This makes the total length of this 1 minute resolution RRA to be 4320 minutes = 3 days

The other RRA stores an average of 5 samples, for a total of 2016 samples. Thus its total duration is 5x2016 min= 10080 min= 7 days.

Thus the default duration of high resolution samples is 3 days and lower resolution samples is 7 days. Tweaking these values will affect both the database size on the client size and the network usage.

For locations with very intermittent connectivity, having bigger-duration databases might be advisable. sample RRA's could be

RRA:AVERAGE:0.5:1:43200 (30 days) RRA:AVERAGE:0.5:5:20160 (70 days)

The above sample values will not be having any impact on network usage, since we're not increasing their resolution, frequency or adding/removing more RRA's

Dumping all information

 * To dump all the information contained in an rrd file, one could use 'rrdtool dump' command.
 * To dump the same info in a more pretty print format (tabular), you may use dsreport. You'll need to install rrdtool-perl to make the script work.

Notes on usage of the actual server/client system
Update: The following version numbers have been tested to work. Previous versions were buggy. Subsequent versions may be good. These are the upstream Fedora, Ubuntu and Debian package repository paths. This is the upstream git repository. The Dextrose-3 operating system will ship the latest stable sugar-client side packages.

Server side packages

 * active-document-0.1-3.1
 * restful-document-0.1-4.1
 * sugar-client-1.2-16.1
 * sugar-stats-client-0.1-6.1
 * sugar-stats-server-0.1-6.1
 * gevent

Client side packages

 * sugar-client-1.2-16.1
 * sugar-stats-client-0.1-6.1

Note: as you may have noticed, the package sugar-stats-client is needed to be installed on both the Client and the Server. The name is a bit misleading as it is more of a library that provides common functionality. This name has/will be changed in future releases.

The configuration files

 * There should be configuration files both on the client and the server. Here are the sample configurations:

Client side config file: /etc/sugar-client.conf
[main] api-url = http://schoolserver:8000 auto-register = 0 auto-restore = False backup-timeout = 0 backup-url = debug = 0 jabber-url = machine-sn = machine-uuid = nickname = schoolserver = stats-ca-certs = /home/anish/Documents/openssl/server.crt stats-timeout = 60 stats-url = https://192.168.1.55:8000 uid = update-args = update-timeout = 0
 * 1) url to connect to school server's RESTfull API
 * 1) process auto-registeration on a school server on startup; multiple argument,
 * 2) if mentioned more than once and previously registered server's IP is different
 * 3) from "schoolserver"'s one, process re-registrtion
 * if, after registration, there are backups to restore on the server, process
 * 1) restore automatically
 * 1) if more than 0, do regular backups with specified delay in seconds
 * 1) rsync url to backup/restore Journal
 * 1) debug logging level; multiple argument
 * 1) jabber server for Sugar collaboraiton
 * 1) specify machine's serial number instead of detecting it
 * 1) specify machine's UUID instead of detecting it
 * 1) specify user's nickname instead of detecting it
 * 1) school server's FQDN, will be auto-set after registration
 * 1) CA certificates file to get access to --stats-url via HTTPS
 * 1) if more than 0 and --stats-url specified, ping stats server with specified
 * 2) delay in seconds to coordinate stats gathering work
 * 1) stats server's http(s) url to interact with; if omited, any stats related
 * 2) functionality is disabled
 * 1) current user's unique identity; needs to be set only from profile
 * 2) configuration file; will be auto-created on initial registration
 * 1) optional command-line arguments to pass to a packager while updating the
 * 2) system
 * 1) if more than 0, do regular unattended updates with specified delay in seconds

Important paramerters to configure are
 * : Path of the SSL certificate in case SSL auth is being used.
 * : The url of the server. This needs to be prefixed with  or   to work without/with SSL.
 * : The sync frequency of the collected statistics between the client and the server. Note, changing this wouldn't typically affect bandwidth consumption, which is actually determined by the RRAs.
 * : Specify these values explicitly if you want them to be automatically determined. Might be useful in making stats anonymous by setting common machine-sn and/or nickname.

Server side config file: /etc/sugar-stats.conf
[stats] stats = True stats-certfile = /home/anish/Documents/openssl/server.crt stats-client-rras = RRA:AVERAGE:0.5:1:4320 RRA:AVERAGE:0.5:5:2016 stats-keyfile = /home/anish/Documents/openssl/server.key stats-root = /var/lib/sugar-stats/rrd stats-server-rras = RRA:AVERAGE:0.5:1:10080 RRA:AVERAGE:0.5:5:8928 RRA:AVERAGE:0.5:15:35040 stats-step = 60
 * 1) enable stats collecting
 * 1) path to SSL certificate to serve stats server via HTTPS
 * 1) space separated list of RRAs for RRD databases on client side
 * 1) path to SSL certificate keyfile to serve stats server via HTTPS
 * 1) path to the root directory to place stats
 * 1) space separated list of RRAs for RRD databases on a server side
 * 1) step interval in seconds for RRD databases

[active-document] data-root = /var/lib/sugar-stats/users find-limit = 32 index-flush-threshold = 32 index-flush-timeout = 60 index-write-queue = 256
 * 1) path to the root directory to place documents' data and indexes
 * 1) limit the resulting list for search requests
 * 1) flush index every specified changes
 * 1) flush index index after specified seconds since the last change
 * 1) for concurent access, run index writer in separate thread; this option
 * 2) specifies the writer's queue size; 0 means not threading the writer

[main] debug = 0 host = 192.168.1.55 logdir = /var/log/sugar-stats port = 8000 rundir = /var/run/sugar-stats trust-users = True
 * 1) debug logging level; multiple argument
 * 1) hostname to listen incomming connections
 * 1) path to the directory to place log files
 * 1) port number to listen incomming connections
 * 1) path to the directory to place pid files
 * 1) switch off user credentials check; disabling this option will require
 * 2) OpenSSH-5.6 or later.

Important paramerters to configure are
 * : Must be true for stats collection to be enabled
 * : Path of the SSL certificate on the server (if SSL auth is being used)
 * : Path of the SSL private key on the server (if SSL auth is being used)
 * : Configuration of Round Robin Archives on the client. See
 * : Configuration of Round Robin Archives on the server. See . This will NOT affect network bandwidth, but merely space consumed on the server.
 * : Step size of ALL RRAs. This WILL affect bandwidth consumption. See
 * : Set this to True if using openSSH version lower than 5.6

Notes on using SSL auth
The steps broadly involved are:
 * Generating private key, and self-signed certificates (for test env). The certificate should be without a passphrase.
 * Copying the certificate on the client, and configure  accordingly.
 * Copying the certificate and private key on the server, and configure  accordingly.

Notes on debugging the SSL auth system
 * Check the  in   configuration is   and not.
 * Check that the certificates are valid or have not expired.
 * To check the SSL side of things, following commands maybe run on the client and the server.
 * Server:
 * Client:  (Replace the ipaddress with the correct server-machine url or IP address.
 * If the process runs smoothly, you should see  at the end on the client side terminal. Fails will accompany an error code which maybe looked up here