Rack Walkthrough

[adamakhtar]

Hey @codereading/readers

as per the [discussion] https://github.com/codereading/HQ/issues/8 in HQ re: structure of codereading sessions, it seems a documented walkthrough of rack would be useful.

So perhaps we can walkthrough rack asking questions as we go along.

So to get thing rolling ill start.

Assuming we are using the rackup command to load an app

i.e.
rackup lobster.ru

then the first place to look would be bin/rackup.ru

where we discover it simply calls

Rack::Server.start

module Rack
  class Server

    # Start a new rack server (like running rackup). This will parse ARGV and
    # provide standard ARGV rackup options, defaulting to load 'config.ru'.
    #
    # Providing an options hash will prevent ARGV parsing and will not include
    # any default options.
    def self.start(options = nil)
      new(options).start
    end

    attr_writer :options

    def initialize(options = nil)
      @options = options
      @app = options[:app] if options && options[:app]
    end

    def options
      @options ||= parse_options(ARGV)
    end

  ....

end

As per the comments, if we dont pass an options hash to Server.start, Rack will parse ARGV for arguments (in this case we passed lobster.rb on the command line). The problem is I don't see how the code above would do this.

start calls new passing nil as the options argument. Within initialize nil is assigned to @options. Any reference to "options" will always refer to the argument so the method

def options
   @options ||= parse_options(ARGV)
end

will never get called. So how does rack call parse_options(ARGV) ???

[adrian-gray]

OK, @RoboDisco, I'll bite... Please correct me if I'm wrong!

When rackup lobster.ru is called, the passed filename is saved in the Ruby global array ARGV and the Rack::Server.start class method is called with no options.

def self.start(options = nil)
  new(options).start
end

Which then builds an instance of Rack with nil options, and calls the start instance method with no parameters. The start instance method checks for options using options, which is not a local variable in the start method

def start &blk
  if options[:warn]
    $-w = true
  end
  ...

so, it uses the options method to return the options

def options
  @options ||= parse_options(ARGV)
end

And, since the @options variable is still nill, it parses the ARGV global from the original call.

Like I said, anyone please feel free to correct me - it's my first go at code reading...

[shioyama]

@sketchy looks good to me, I'll pick up from there.

Here's the parse_options instance method, which is called in the options method above with the parameter ARGV:

def parse_options(args)
  options = default_options

  # Don't evaluate CGI ISINDEX parameters.
  # http://hoohoo.ncsa.uiuc.edu/cgi/cl.html
  args.clear if ENV.include?("REQUEST_METHOD")

  options.merge! opt_parser.parse!(args)
  options[:config] = ::File.expand_path(options[:config])
  ENV["RACK_ENV"] = options[:environment]
  options
end

The first thing that happens is that options is assigned the return value of the instance method default_options:

def default_options
  {
    :environment => ENV['RACK_ENV'] || "development",
    :pid => nil,
    :Port => 9292,
    :Host => "0.0.0.0",
    :AccessLog => [],
    :config => "config.ru"
  }
end

The options above are described in the comments.

After setting the defaults, options is then merged with the arguments in ARGV, which are parsed by the object returned from the method opt_parser. If you look up opt_parser, you'll see it's very simple:

def opt_parser
  Options.new
end

Options is a class with just two functions, parse! and handler_opts. Here parse! is being called with ARGV as args.

def parse!(args)
  options = {}
  opt_parser = OptionParser.new("", 24, ' ') do |opts|

...

  end

  begin
    opt_parser.parse! args
  rescue OptionParser::InvalidOption => e
    warn e.message
    abort opt_parser.to_s
  end

  options[:config] = args.last if args.last
  options
end

Options uses ruby's OptionParser class to actually parse the arguments (the details of which are not very relevant to understanding rack).

After parsing the options from ARGV, parse_options expands the path of the config filename ("config.ru" in our example), then assigns the environment variable RACK_ENV to whatever has been set in the options, and finally returns the value of the local variable options.

A couple things I'm not sure about here. Anyone know what this line is about? (The link is broken but you can find it on archive.org.)

  # Don't evaluate CGI ISINDEX parameters.
  # http://hoohoo.ncsa.uiuc.edu/cgi/cl.html
  args.clear if ENV.include?("REQUEST_METHOD")

And also, it's not really clear to me why you need to set the RACK_ENV environment variable from the options in parse!, but I guess that will become clear as we move on further through the code.

Hope that makes sense!

[adamakhtar]

@sketchy Thanks for that - I must have had too much coffee that day. I was mistaking one instance start method for the class start method.

@shioyama Rackup sets the environment because later on it automatically adds some useful middleware to the app. What gets added depends on the environment specified.

For :development CommonLogger, ShowExceptions and Lint are added.
For :production only CommonLogger
For :none nothing will be added automatically.

So I guess somewhere down the line we will see the value of RACK_ENV being checked and then corresponding middleware being inserted.

is that what you meant by your question?

Thank you both for the write up as well!

[shioyama]

@RoboDisco Yes that's exactly what I meant, thanks!

[adamakhtar]

continuing on we come back to the start method mentioned above by @sketchy

which looks like ( with some omissions for brevity sake)

def start &blk
      #process options here 
      ...
      wrapped_app
      ...
      server.run wrapped_app, options, &blk
end

examining wrapped app we see

def wrapped_app
  @wrapped_app ||= build_app app
end

it calls build_app passing it the results of the method (attr reader) app

def app
      @app ||= begin

        if !::File.exist? options[:config]
          abort "configuration #{options[:config]} not found"
        end

        app, options = Rack::Builder.parse_file(self.options[:config], opt_parser)
        self.options.merge! options
        app
      end
end

where it lazily loads the app. To do so it first makes sure the specified rack config file exists. Then it parses the config file i.e. lobster.ru.

It appears that the app options could change in the config file so it updates it's options hash with the one returned by the parse.

Should we head to the builder file? should we open a new issue for that file or just continue on?

[featureenvy]

Now this line bugged me for a long time (thanks @shioyama ;)):

# Don't evaluate CGI ISINDEX parameters.
# http://hoohoo.ncsa.uiuc.edu/cgi/cl.html
args.clear if ENV.include?("REQUEST_METHOD")

I think i have figured out why this is here. It's hard to find documentation on this, because that feature is very rarely used (or not at all in 2012). Though I found another source (chapter 9.3.2).

Normally a CGI query would look something like this: http://www.myserver.com/cgi-bin/echo?content=this. If the query parameter on the other hand doesn't contain a = (like http://www.myserver.com/cgi-bin/finger?root), it will be passed to the script as a command line argument. So in this case it would execute finger root.

Now of course Rack isn't CGI, but it's CGI-like. To prevent a server from executing rack directly from the command line, they check if a REQUEST_METHOD is given in the ENV, in which case the request came from the network and not directly from the command line. So that's why they wipe the args if the env includes REQUEST_METHOD.

Does that happen in practice? My guess is no, I don't think Unicorn or Passenger would implement something like this. But better to be safe than sorry.

[shioyama]

@Zumda thanks for digging up that info, interesting to know!