Introducing Comfygure, A Configuration Manager For Easy Deployments

TL;DR: We've just published Comfygure, a tool facilitating the sharing of configuration settings between developers and / or environments. Comfygure assumes that you deploy artefacts that require settings to run in various environment. Comfygure solves the problem of managing, storing, and deploying these settings. It features end-to-end encryption, configuration versioning (git-like), and supports multiple formats (JSON, YML, environment variables). Now in beta, it's available on GitHub marmelab/comfygure. Give it a try!

The Problem: Managing Application Settings

As a developer, I usually work on several projects at the same time. When I'm lucky, a given project only runs on two environments (development and production), but more often than not, I need to handle additional environments (CI, staging, stress test, etc.) Even worse, I sometimes work with more than one workstation (a Linux for development, and a Mac for Apple Store publication). Each of these project instances needs a specific set of configuration settings for API access, database credentials, certificates, logging level, etc.

There is no silver bullet for configuration management and deployment, but it's generally accepted that a deployment artifact should be configured at the very last moment, before the deployment. In this way, I can use the same artifact to deploy on integration, production or any other environment. To achieve "The Right Deployment Process™", I usually write a configuration file (or a list of environment variables) for each environment, workstation, continuous delivery system, and server instance.

// in config/development.js
module.exports = {
  api: {
    port: 3000,
    protocol: "http",
  },
  prismic: {
    endpoint: "https://acme.cdn.prismic.io/api",
    accessToken: "Ft5yDE3685duHfZWSiy7fgd46",
    release: "master",
  },
  batch: {
    apiKey: "345635675677434746858797897990",
    token: "4356345745685678567679679679768",
  },
  push_web_notification: {
    provider: "pusher",
    app_id: "A3452345",
    app_key: "45645624564357567",
    app_secret: "2624645624564356",
  },
  healthcare: {
    login: "34568465856784",
    password: "S3cr3t",
  },
  // etc...
};

We never commit these configuration files together with the source code. It's a bad practice for security reasons. Otherwise, that means that anyone with access to the source code also has access to all the environments (including production database, etc). So each server or build system has a slightly different copy of the configuration file. And it's a pain in the neck to update these files each time I need to add a new setting. And for some environments, I can't do it myself. For instance, I don't have access to the production environment, so I must ask the sysadmins to change the environment variables there.

I don't count the failed builds and deployments due to a forgotten config key, and I'm tired of asking a bunch of configuration files to my co-worker the first time I install a project.

The Solution: A Configuration Manager

The solution to the problem of configuration management is... a configuration manager (or a secret management tool). It's similar to a team password manager in terms of productivity, but for configs.

There a many existing secret management tools, including:

• Ansible Vault
• Barbican
• Chef Data Bags
• Chef Vault
• Citadel
• Confidant
• Configuration Storage Systems (Consul, etcd, Zookeeper)
• Conjur
• Crypt
• EJSON
• Keywhiz
• Knox
• Red October
• Trousseau
• Vault (Hashicorp)

All the configuration managers we could find are either tied to a huge ecosystem (like AWS or Atlassian), too complex to set up, too focused on secret management, not open-source, or don't really fullfil our needs.

Why We Chose To Write A New One

We searched for a solution for several weeks, postponing the question regularly, but at the end we had to face the fact that there is no standard tool for our simple need. We even asked specialists in the field, but the solution they found is to hire a Release Manager to deal with deployment issues. Neither we nor our clients want such a thing.

We really try to be pragmatic developers, and don't want to reinvent the wheel each time we face a problem. But when we have the same problem over and over, and we don't find an efficient solution, we roll up our sleeves and make things work.

Also, we love both challenge, and sharing knowledge with our pairs. So we wrote Comfygure. Unlike many other Secret Management Tools, comfygure doesn't try to pack too many features into one tool. Comfygure tries to do one thing (settings deployment), and do it well.

How Comfygure works

Comfygure proposes to solve configuratino management by storing settings on a remote server (like a remote git), encrypted. Comfygure clients know how to read and write from that remote server, and decrypt the settings to dump it locally.

Ultimately, this lets you execute the following command from any server:

> comfy get development --envvars
export LOGIN='admin';
export PASSWORD='S3cr3T';

Usage

Comfygure comes with a console command: comfy. Run it in a project folder to bootstrap a comfygure project.

npm install -g comfy
cd myproject
comfy init
comfy env add production

Once Comfygure is initialized for a directory, you can send the configuration for that project in a given environment to the comfy server.

comfy setall production app/config/production.json

The configuration is be encrypted and signed on the client side - the comfy server won't be able to read it. The comfy server can be in your own infrastructure, or in the cloud - we provide a Serverless implementation, which can run on AWS lambda.

On your CI/CD, export the comfy credentials (available in .comfy/config), and run the following command to retrieve your config. Again, it will be decrypted and checked client side.

comfy get production --envvars | source /dev/stdin

The additional features are explained on the documentation. You can also find a more complete explanation of how comfygure works internaly.

What's Next

We are confident enough with Comfygure to use it on production to deploy our customers' projects.

Today, we open a public beta! We offer an optional cloud storage for your tests (free for the beta period), but you should probably host your own comfy server. Comfygure is built in Node.js, and it's open-source (MIT License)

Feel free to open an issue on GitHub if you find a bug, or if you want to suggest something. We are raring to get your feedback about it, and your proposals to make it better. Also, you can ask your questions on Stack Overflow.

Meanwhile, we will monitor our comfy server, and continue to develop new features, like the ability the manage configuration branches with tags (git-like) for example. You can follow the next developments on GitHub marmelab/comfygure.