Infrastructure Team/Puppet: Difference between revisions

(20 intermediate revisions by 2 users not shown)

Line 1:

~~{{TOCright}}~~

Sugar Labs Puppet infrastructure.

~~Sugar Labs Puppet infrastructure.~~

== Modules ==

~~== Run stages ==~~

Puppet modules configure particular services like [[Service/git|git.sugarlabs.org]] or MySQL but not tied to final configurations, they are being configured from [[#Master|puppetmaster]].

[http://~~docs~~.~~puppetlabs~~.~~com/guides~~/~~language_tutorial.html#run_stages Run stages~~] ~~were introduced in Puppet v2~~.~~6. Within~~ Sugar Labs ~~infrastructure there~~ are ~~additional, to default Puppet, stages. All puppet modules should take care about it~~.

All modules are collected as repositories in [http://git.sugarlabs.org/puppets puppets] Gitorious project. Modules might be created from scratch or mirrored from upstream, so, all modules that are used within Sugar Labs are stored in one place. Module repository might have followed branches:

~~Defined stages:~~

* ''master'', development version (might be absent if project is mirrorred from upstream),

* ''production'', version is being used in production,

* ''upstream'', if it is mirrorred project.

* ''initialize'' only for high-level initialization on nodes level,

Auto generated [http://api.sugarlabs.org/puppets/ documentation].

* ''main'' default Puppet stage to run nodes.

~~Module resources should be registered after ''initialize'' stage and might be before ''main'' one.~~

== Master ==

~~== Low-level modules ==~~

Only puppetmaster keeps final configuration for all Sugar Labs services. It is being started in a private git repository on [[Machine/lightwave]], which is a high security machine.

Modules to maintain particular applications that don't contain any data tied to end-usage environment, all configuration happen via class/define/type arguments from out of these modules. Not all module repositories might be hosted on git.sugarlabs.org.

=== Private repository ===

~~'''[http:~~//git~~.sugarlabs.org~~/~~projects~~/puppet~~-util util]''' ~~

* {{Code|/var/lib/puppet/etc/puppet.git}}

~~Common infrastructure routines. Module contains only code, not any data~~.

* {{Code|/var/lib/puppet/etc/puppet}} detached working directory

* {{Code|/etc/puppet}} symlink to detached working directory

* {{Code|/var/lib/puppet/etc/puppet.git/config}}:

~~'''~~[~~http:~~//~~git~~.~~sugarlabs~~.~~org/projects/puppet-firewall firewall~~]~~''' ~~

[core]

~~High level setup of iptables.~~

repositoryformatversion = 0

filemode = true

bare = false

sharedRepository = true

logallrefupdates = true

worktree = /etc/puppet

[receive]

denycurrentbranch = ignore

[hooks]

mailinglist = systems-logs@...

emailprefix = "[PUPPET] "

showrev = "git show -C %s; echo"

~~'''[http:~~//~~git.sugarlabs.org~~/~~projects~~/puppet~~-mysql mysql]''' ~~

* {{Code|/var/lib/puppet/etc/puppet.git/description}}:

~~MySQL procedures~~.

~~'''[http://git.sugarlabs.org/projects/puppet-lighttpd lighttpd]''' ~~

Sugar Labs Puppet configuration

~~Lighttpd support.~~

~~'''[http:~~//~~github~~.~~com~~/~~reductivelabs~~/~~puppet~~-~~vcsrepo vcsrepo]''' ~~

* {{Code|/var/lib/puppet/etc/puppet.git/hooks/post-receive}}

~~Version control systems procedures.~~

~~'''[http:~~//git~~.sugarlabs.org~~/~~projects~~/~~puppet~~-~~supybot supybot]''' ~~

#!/bin/bash

~~Setup supybot IRC bot~~.

sh /usr/share/doc/git-core/contrib/hooks/post-receive-email

git checkout -f

for i in $(ls /etc/puppet/manifests/nodes/*.pp); do puppet kick $(basename $i .pp); done

== ~~Sugar Labs specific modules~~ ==

=== Repository hierarchy ===

Modules to maintain applications that make sense only in Sugar Labs environment. Modules don't contain any data tied to end-usage environment, all configuration happen via class/define/type arguments from out of ~~these modules. All module repositories are hosted on git.sugarlabs.org.~~

Git repository consists of:

~~'''[http:/~~/~~git~~.~~sugarlabs~~.~~org~~/~~projects~~/~~puppet-bazaar bazaar]''' ~~

* {{Code|manifests/site.pp}}, main recipe which contain common settings and includes nodes.

~~Maintain bazaar~~.~~sugarlabs.org infrastructure~~.

* {{Code|manifests/services/}}, recipes that describes final configuration for particular services.

* {{Code|manifests/nodes/}}, puppetized nodes, configuration for hosts that ask puppetmaster.

* {{Code|modules/}}, git submodules with Puppet [[#Modules|modules]].

== Nodes ==

~~''In progress''~~

Hosts that fetch configuration from the [[#Master|puppetmast]] need <code>puppet >= 2.6</code> package.

If {{Code|puppet}} was installed from gems, it needs to be initially configured:

puppet master --mkusers

To complete configuration, execute:

~~End~~-~~usage nodes~~. ~~Configuration settings are tracked by local git repository only since contain private data~~.

puppet agent --no-daemonize --onetime --server puppet.sugarlabs.org

~~=== jita~~.~~sugarlabs~~.~~org ===~~

Every puppetized host should have particular node file in {{Code|manifests/nodes/<host-name>.pp}} on [[#Master|puppetmaster]].

@@ Line 1: / Line 1: @@
-{{TOCright}}
+Sugar Labs Puppet infrastructure.
-Sugar Labs Puppet infrastructure.
+== Modules ==
-== Run stages ==
+Puppet modules configure particular services like [[Service/git|git.sugarlabs.org]] or MySQL but not tied to final configurations, they are being configured from [[#Master|puppetmaster]].
-[http://docs.puppetlabs.com/guides/language_tutorial.html#run_stages Run stages] were introduced in Puppet v2.6. Within Sugar Labs infrastructure there are additional, to default Puppet, stages. All puppet modules should take care about it.
+All modules are collected as repositories in [http://git.sugarlabs.org/puppets puppets] Gitorious project. Modules might be created from scratch or mirrored from upstream, so, all modules that are used within Sugar Labs are stored in one place. Module repository might have followed branches:
-Defined stages:
+* ''master'', development version (might be absent if project is mirrorred from upstream),
+* ''production'', version is being used in production,
+* ''upstream'', if it is mirrorred project.
-* ''initialize'' only for high-level initialization on nodes level,
+Auto generated [http://api.sugarlabs.org/puppets/ documentation].
-* ''main'' default Puppet stage to run nodes.
-Module resources should be registered after ''initialize'' stage and might be before ''main'' one.
+== Master ==
-== Low-level modules ==
+Only puppetmaster keeps final configuration for all Sugar Labs services. It is being started in a private git repository on [[Machine/lightwave]], which is a high security machine.
-Modules to maintain particular applications that don't contain any data tied to end-usage environment, all configuration happen via class/define/type arguments from out of these modules. Not all module repositories might be hosted on git.sugarlabs.org.
+=== Private repository ===
-'''[http://git.sugarlabs.org/projects/puppet-util util]'''<br>
+* {{Code|/var/lib/puppet/etc/puppet.git}}
-Common infrastructure routines. Module contains only code, not any data.
+* {{Code|/var/lib/puppet/etc/puppet}} detached working directory
+* {{Code|/etc/puppet}} symlink to detached working directory
+* {{Code|/var/lib/puppet/etc/puppet.git/config}}:
-'''[http://git.sugarlabs.org/projects/puppet-firewall firewall]'''<br>
+ [core]
-High level setup of iptables.
+ repositoryformatversion = 0
+ filemode = true
+ bare = false
+ sharedRepository = true
+ logallrefupdates = true
+ worktree = /etc/puppet
+ [receive]
+ denycurrentbranch = ignore
+ [hooks]
+ mailinglist = systems-logs@...
+ emailprefix = "[PUPPET] "
+ showrev = "git show -C %s; echo"
-'''[http://git.sugarlabs.org/projects/puppet-mysql mysql]'''<br>
+* {{Code|/var/lib/puppet/etc/puppet.git/description}}:
-MySQL procedures.
-'''[http://git.sugarlabs.org/projects/puppet-lighttpd lighttpd]'''<br>
+ Sugar Labs Puppet configuration
-Lighttpd support.
-'''[http://github.com/reductivelabs/puppet-vcsrepo vcsrepo]'''<br>
+* {{Code|/var/lib/puppet/etc/puppet.git/hooks/post-receive}}
-Version control systems procedures.
-'''[http://git.sugarlabs.org/projects/puppet-supybot supybot]'''<br>
+ #!/bin/bash
-Setup supybot IRC bot.
+ sh /usr/share/doc/git-core/contrib/hooks/post-receive-email
+ git checkout -f
+ for i in $(ls /etc/puppet/manifests/nodes/*.pp); do puppet kick $(basename $i .pp); done
-== Sugar Labs specific modules ==
+=== Repository hierarchy ===
-Modules to maintain applications that make sense only in Sugar Labs environment. Modules don't contain any data tied to end-usage environment, all configuration happen via class/define/type arguments from out of these modules. All module repositories are hosted on git.sugarlabs.org.
+Git repository consists of:
-'''[http://git.sugarlabs.org/projects/puppet-bazaar bazaar]'''<br>
+* {{Code|manifests/site.pp}}, main recipe which contain common settings and includes nodes.
-Maintain bazaar.sugarlabs.org infrastructure.
+* {{Code|manifests/services/}}, recipes that describes final configuration for particular services.
+* {{Code|manifests/nodes/}}, puppetized nodes, configuration for hosts that ask puppetmaster.
+* {{Code|modules/}}, git submodules with Puppet [[#Modules|modules]].
 == Nodes ==
-''In progress''
+Hosts that fetch configuration from the [[#Master|puppetmast]] need <code>puppet >= 2.6</code> package.
+If {{Code|puppet}} was installed from gems, it needs to be initially configured:
+ puppet master --mkusers
+To complete configuration, execute:
-End-usage nodes. Configuration settings are tracked by local git repository only since contain private data.
+ puppet agent --no-daemonize --onetime --server puppet.sugarlabs.org
-=== jita.sugarlabs.org ===
+Every puppetized host should have particular node file in {{Code|manifests/nodes/<host-name>.pp}} on [[#Master|puppetmaster]].