Lotus Apps: How to Manage Environment Variable in Ruby Subprocess

I recently worked on a project that caused some puzzlement when spawning subprocesses multiple times, all while continuing to use bundler to manage our gems. I needed to find an easy way to manage environment variable in ruby using bundler, NodeJS and rbenv.

The Problem

We have a custom node app that runs acting as a coordinator to executed arbitrary tasks (most of them written in ruby) to do things such as generate screenshots for an iOS application.

The node app itself contains the ruby scripts in the same directory, which uses Bundler to manage gems. However, when node executes a task it does that in a unique folder in the system tmp directory to execute many tasks at a time on different data sets.

Ultimately we wanted to use rbenv and bundler to manage ruby and gems, but yet needed that to work from any arbitrary path on the filesystem (outside the directory containing the ruby code and the Gemfile).

Example Filesystem

Here’s an example situation that illustrates the issue:

- ~/Documents/node_app/
  - .ruby-version
  - app.js
  - Gemfile
  - tasks/
    - custom_script.rb
- /tmp/
  - unique_task_run

In the above setup, the “app.js” defines the running NodeJS app which would execute “custom_script.rb” via a subprocess like the following:

var exec = Promise.promisify(require('child_process').exec)
return exec("custom_script.rb", {
  env: extend({
    CUSTOM_ENV: "value"
  }, process.env)
});

In the above code, we configure the environment variables to pass to the ruby script by including everything currently in our environment along with an extra value. This executes fine, until we execute a task from within the ruby script, such as:

#!/usr/bin/env ruby
Dir.chdir('/tmp') do
  Dir.chdir('some_new_folder')
  result = `executable_from_gemfile`
end

Ruby ENV

Our ruby script has environment variables set from the running Node process, which is great. The current script is even located within the project, so there is a Gemfile nearby. However, when we execute a new script inside our ruby script, we’re no longer in the node directory — we’re in the “tmp” directory working on a dataset.

There are many different ways of executing tasks in ruby, and we’ll be writing another post on that very subject later on. For now, let’s just review the documentation to know that using backticks executes the task in a new shell and passes along everything in the current ENV to the new task.

Observant readers would have undoubtably already noticed that we’re not calling our executable using bundler. So let’s try this:

result = `bundle exec executable_from_gemfile`

While you would think this would work, we receive an error because bundler cannot find our Gemfile.

Could not locate Gemfile or .bundle/ directory

Bundler Gemfile Environment Variable

We’re going to solve this problem by working back up the call tree of processes. First off, let’s observe that changing our ruby script to the following fixes our issue:

result = `BUNDLE_GEMFILE=~/Documents/node_app/Gemfile bundle exec executable_from_gemfile`

By setting this environment variable, bundler can locate the Gemfile and run the executable provided by a gem in our bundle just fine. However, it’s cumbersome to provide this info every time and error prone.

Refactoring to introduce a method to prefix the environment to a command would be a good step in the right direction, but feels a little clunky.

Remember, ENV is inherited

If we recall that the ENV is inherited for a subprocess, then ensuring that the parent process executing our “custom_script.rb” provides us with this information is the next step. In our example, the node process is our parent.

# node
exec("custom_script.rb", {
  env: extend({
    "BUNDLE_GEMFILE": "~/Documents/node_app/Gemfile"
   }, process.env)
})

By specifying the environment variable when we execute the ruby script from node, we’ve ensured that any tasks executed can in turn run a subprocess of their own and use “bundle exec” properly, no matter the current directory the task is running from.

One More Level

Setting the “BUNDLE_GEMFILE” setting in our node codebase isn’t good practice either, and could vary by installation if we deploy the same node app to different servers. One easy way to define this would be an environment variable when starting the node app. We were already using the wonderful dotenv package, so adding this was easy.

# .env
BUNDLE_GEMFILE=~/Documents/node_app/Gemfile

Wrapping Up

Our final solution boils down to this:

# .env
BUNDLE_GEMFILE=~/Documents/node_app/Gemfile
# app.js
exec("custom_script.rb", {
  env: extend({
    CUSTOM_VAR: "value"
  }, process.env)
})
# custom_script.rb
result = `bundle exec executable_from_gemfile`

We’ve ensured that any subprocesses spawned by ruby scripts, executed from our node app, properly inherit environment variables and can use “bundle exec” with ease. Additionally, since we’re using the “shims” feature of rbenv, the “bundle” command itself is properly linked to our version of ruby no matter the directory we’re executing from.

How to Manage Environment Variable in Ruby Subprocess