deployment - Managing application configuration in a chef environment cookbook -

i new chef , have been struggling find best practices on how configure application configuration in environment cookbook [source #1].

the environment cookbook i'm working on should following:

• prepare node custom application deployment creating directories, users, etc. specific deployment only.
• add initialization , monitoring scripts specific application deployment.
• define application configuration settings.

this last responsibility has been particularly tough nut crack.

an example configuration file of application deployment might follows:

{     "server": {         "port": 9090     },     "session": {         "proxy": false,         "expires": 100     },     "redis": [{         "port": 9031,         "host": "rds01.prd.example.com"     }, {         "port": 9031,         "host": "rds02.prd.example.com"     }],     "ldapconfig": {         "url": "ldap://example.inc:389",         "admindn": "cn=admin,cn=users,dc=example,dc=inc",         "adminusername": "user",         "adminpassword": "secret",         "searchbase": "ou=bigcustomer,ou=customers,dc=example,dc=inc",         "searchfilter": "(example=*)"     },     "log4js": {         "appenders": [             {                 "category": "[all]",                 "type": "file",                 "filename": "./logs/myapp.log"             }         ],         "levels": {             "[all]": "error"         }     },     "otherservice": {         "basepath" : "http://api.prd.example.com:1234/otherservice",         "smoketestvariable" : "testvar"     } } 

some parts of deployment configuration file more stable others. while may vary depending on application , setup, things port numbers , usernames prefer keep same across environments simplicity's sake.

let me classify configuration settings:

stable properties

• session
• server
• log4js.appenders
• ldapconfig.adminusername
• ldapconfig.searchfilter
• otherservice.basepath
• redis.port

environment specific properties

• log4js.levels
• otherservice.smoketestvariable

partial-environment specific properties

• redis.host: rds01.[environment].example.com
• otherservice.basepath: http://api.[environment].example.com:1234/otherservice

encrypted environment specific properties

• ldapconfig.adminpassword

questions

1. how should create configuration file? options: 1) use file shipped within application deployment itself, 2) use cookbook file template, 3) use json blob 1 of attributes [source #2], 4)... other?
2. there great diversity of variability in configuration file; how best manage these using chef? roles, environments, per-node configuration, data-bags, encrypted data-bags...? or should opt environment variables instead?

some key concerns in approach:

• i prefer there 1 way set configuration settings.
• changing configuration file developer should straightforward (they using vagrant on local machines before pushing test).
• the passwords must secure.
• the chef cookbook managed within same git repository sourcecode.
• some configuration settings require great deal of flexibility; example log4js setting in example config might contain many more appenders dozens of unstructured variables.

any experiences appreciated!

sources

1. http://blog.vialstudios.com/the-environment-cookbook-pattern/
2. http://lists.opscode.com/sympa/arc/chef/2013-01/msg00392.html
3. http://jtimberman.housepub.org/blog/2013/01/28/local-templates-for-application-configuration/
4. http://realityforge.org/code/2012/11/12/reusable-cookbooks-revisited.html

jamie winsor gave talk @ chefconf goes further in explaining environment cookbook pattern's rationale , usage:

• chefcon: talking self-contained releases, using chef
• slides

in opinion 1 of key concepts pattern introduces idea of using chef environments control settings of each application instance. environment updated, using berkshelf, run-time version of cookbooks being used application.

what less obvious if decide reserve chef environment use of single application instance, becomes safe use environment configure application's global run-time settings.

an example if given in berkshelf-api installation instructions. there see production environment (for application) being edited various run-time settings:

knife environment edit berkshelf-api-production 

in conclusion, chef gives lots of options. make following generic recommendations:

1. capture defaults in application cookbook
2. create environment each application instance (as recommended pattern)
3. set run-time attribute over-rides in environment

notes:

• see berksflow tool. designed make environment cookbook pattern easier implement.
• i have made no mention of using roles. these can used override attributes @ run-time, might simpler capture in dedicated chef environment. roles seem better suited capturing information peculiar component of application.