Refactoring a large case statement

February 3, 2019
ruby

I'm currently building a gem for converting curl commands to something else, and one early problem I needed to deal with was parsing the curl command arguments.

The basic idea for a gem is to convert curl command to a sweet looking hash that we can use to do magical things. So when given:

curl https://isrubydead.com/

Our program converts it to:

{
  "body": "",
  "method": "GET",
  "headers": {},
  "url": "https://isrubydead.com/"
}

We'll call this hash a Request, so let's define it:

Request = Struct.new(:body, :method, :headers, :url) do
  def initialize
    self.method = 'GET'
    self.headers = {}
    self.url = ''
    self.body = ''
  end
end

Our program should expect the curl command as a command line argument, so let's use ARGV.first to read the curl command:

require 'shellwords'

module ParseCurl
  class << self
    def parse(command)
      self.request = Request.new
      self.parser = Parser.new(request)

      parser.parse(Shellwords.split(command.strip))

      request.to_h
    end

    private

    attr_accessor :request
    attr_accessor :parser
  end
end

ParseCurl.parse(ARGV.first)

We use Shellwords here to convert the curl command to an array of tokens, to make it easier to parse.

Now that we have a place to store the result of parsing the curl command, and we are passing the command line argument to a parser, it's time to write an actual parser.

Being a very serious and very senior developer, I've ended up with this very appealing class:

class Parser
  def initialize(request)
    @request = request
  end

  def parse(words)
    self.words = words

    while (word = words.shift)
      parse_word(word)
    end
  end

  private

  attr_accessor :words
  attr_reader :request

  def parse_word(word)
    case word
    when URI::DEFAULT_PARSER.make_regexp
      request.url = word
    when /\A(-H|--header)\Z/
      parse_header
    when /\A--compressed\Z/
      request.headers['Accept-Encoding'] ||= 'deflate, gzip'
    when /\A-X|\A--request\Z/
      request.method = extract_method(word)
    when /\A(-u|--user)\Z/
      request.headers['Authorization'] ||=
        "Basic #{Base64.strict_encode64(words.shift)}"
    when /\A(-b|--cookie)\Z/
      request.headers['Set-Cookie'] ||= words.shift
    when /\A(-A|--user-agent)\Z/
      request.headers['User-Agent'] ||= words.shift
    when /\A(-I|--head)\Z/
      request.method = 'HEAD'
    when /\A(-d|--data|--data-ascii)\Z/
      parse_data
    when /\A--data-binary\Z/
      parse_data(strip_newlines: false)
    when /\A--data-raw\Z/
      parse_data(raw: true)
    when /\A--data-urlencode\Z/
      parse_data(url_encode: true)
    when /\./
      request.url = "http://#{word}" if request.url == ''
    end
  end
end

I know, a thing a beauty.

What's also very nice is that this class also have to know everything about parsing curl arguments; from header and method arguments to various data arguments.

Super nice! Just look at these beautiful methods, that all belong to this class:

class Parser
  private

  def parse_data(**args)
    request.method = 'POST' if request.method == 'GET' ||
                               request.method == 'HEAD'
    request.headers['Content-Type'] ||=
      'application/x-www-form-urlencoded'

    data = extract_data(args)

    request.body = if request.body.empty?
                     data
                   else
                     "#{request.body}&#{data}"
                   end
  end

  def parse_header
    header = extract_header(words.shift)

    return unless header

    request.headers[header.keys.first] =
      header.values.first
  end

  def extract_data(args)
    data = words.shift
    data = read_file(data, args) if data =~ /\[email protected]/ || args[:raw]
    data
  end

  def read_file(word,
                strip_newlines: true,
                raw: false,
                url_encode: false)
    path = raw ? word : word[1..-1]
    content = File.read(path)
    content.gsub!(/\n|\r/, '') if strip_newlines
    content = CGI.escape(content) if url_encode
    content
  rescue Errno::ENOENT
    file_path = File.expand_path(data[1..-1])
    raise FileNotFoundError,
      "Cannot read file: #{file_path}. Does it exist?"
  end

  # Needed for extracting -XPUT and similar formats.
  def extract_method(word)
    return words.shift unless word =~ /^-X/

    if !(method = word.match(/^-X(.*)/)[1]).empty?
      method
    else
      words.shift
    end
  end

  def extract_header(header)
    [header.split(/: (.+)/)].to_h
  rescue StandardError
    nil
  end
end

Marvelous! This class knows absolutely everything about curl and all the arguments and options. We could ask it the curl's shoe size, and it would give us the answer.

Now that we have thought this class so much, and it has achieved the master's degree worth of education in the University of curl, I'm a little worried it's going to take over the world.

Sure, it only knows about curl for now, but what happens when someone decides it should know about, I don't know, some scary AI thing. It looks innocent right now, but then you turn around and it gets interested in doors.

I don't trust it one bit, so let's steal some knowledge from this class and let's spread it to other classes.

I'll now start explaining the refactoring step by step, but you can take a look at the final result if you'd like to skip.

This class will from now on be ignorant of curl arguments, so the humongous case statement becomes:

class Parser
  def parse_word(word)
    Word.parse(request, words, word)
  end
end

Since every scenario we have to cover in the case statement consists of the regex check and the parsing part, the idea is to abstract those into class.

Let's introduce the Word::Base class:

module Word
  class Base
    # Used to check if the word given matches the subclass' regex.
    def self.===(word)
      word =~ regex
    end

    def self.regex
      raise NotImplementedError, "#{name} doesn't have .regex defined"
    end

    def parse(*)
      raise NotImplementedError, "#{self.class.name} doesn't have #parse defined"
    end
  end
end

The plan is to have one base class, that will be inherited for every scenario we'd like to cover. Here's the first example of one such class:

module Word
  class Header < Base
    def self.regex
      /\A(-H|--header)\Z/
    end

    def parse(*)
      header = extract_header(words.shift)

      return unless header

      request.headers[header.keys.first] = header.values.first
    end

    private

    def extract_header(header)
      [header.split(/: (.+)/)].to_h
    rescue StandardError
      nil
    end
  end
end

This class stores information about everything that's needed to parse a header argument. It detects the argument (when the curl command includes -H or --header in arguments) and then changes the request hash, which we return in the end.

Since this class has to know about words and request too, we'll add that to the base class. We expect other descendants to need that too:

module Word
  class Base
    def initialize(words, request)
      @words = words
      @request = request
    end

    private

    attr_reader :words
    attr_reader :request
  end
end

The next problem we have is that there's no connection between the original parse_word method and this new class we have created, so let's write a method that connects those two:

module Word
  def self.parse(request, words, word)
    subclass = Word::Base.subclasses
                         .detect { |klass| klass === word }
    subclass&.new(words, request)&.parse(word)
  end

  class Base
    # List of subclasses, first-inherited class first.
    def self.subclasses
      ObjectSpace.each_object(Base.singleton_class)
                 .reject { |klass| klass == Base }.reverse
    end
  end
end

With this method we can find a Word::Base subclass that matches the word that was given and then parses that word. If we find the subclass, we try to parse the word. So our current code works for parsing --header or -h words from curl.

Our code now supports parsing headers, so let's proceed to add classes for every argument we'd like to support:

module Word
  class Compressed < Base
    def self.regex
      /\A--compressed\Z/
    end

    def parse(*)
      request.headers['Accept-Encoding'] ||= 'deflate, gzip'
    end
  end

  class Request < Base
    def self.regex
      /\A-X|\A--request\Z/
    end

    def parse(word)
      request.method = extract_method(word)
    end

    private

    # Needed for extracting -XPUT and similar formats.
    def extract_method(word)
      return words.shift unless word =~ /^-X/

      if !(method = word.match(/^-X(.*)/)[1]).empty?
        method
      else
        words.shift
      end
    end
  end

  class Authorization < Base
    def self.regex
      /\A(-u|--user)\Z/
    end

    def parse(*)
      request.headers['Authorization'] ||=
        "Basic #{Base64.strict_encode64(words.shift)}"
    end
  end

  class Cookie < Base
    def self.regex
      /\A(-b|--cookie)\Z/
    end

    def parse(*)
      request.headers['Set-Cookie'] ||= words.shift
    end
  end

  class UserAgent < Base
    def self.regex
      /\A(-A|--user-agent)\Z/
    end

    def parse(*)
      request.headers['User-Agent'] ||= words.shift
    end
  end

  class Head < Base
    def self.regex
      /\A(-I|--head)\Z/
    end

    def parse(*)
      request.method = 'HEAD'
    end
  end

  class Data < Base
    def self.regex
      /\A(-d|--data|--data-ascii)\Z/
    end

    def options
      {}
    end

    def parse(*)
      request.method = 'POST' if request.method == 'GET' ||
                                 request.method == 'HEAD'
      request.headers['Content-Type'] ||=
        'application/x-www-form-urlencoded'

      data = extract_data(options)

      request.body = if request.body.empty?
                       data
                     else
                       "#{request.body}&#{data}"
                     end
    end

    def extract_data(args)
      data = words.shift
      data = read_file(data, args) if data =~ /\[email protected]/ ||
                                      args[:raw]
      data
    end

    def read_file(word,
                  strip_newlines: true,
                  raw: false,
                  url_encode: false)
      path = raw ? word : word[1..-1]
      content = File.read(path)
      content.gsub!(/\n|\r/, '') if strip_newlines
      content = CGI.escape(content) if url_encode
      content
    rescue Errno::ENOENT
      file_path = File.expand_path(data[1..-1])
      raise FileNotFoundError,
            "Cannot read file: #{file_path}. Does it exist?"
    end
  end

  class StripNewlinesData < Data
    def self.regex
      /\A--data-binary\Z/
    end

    def options
      { strip_newlines: false }
    end
  end

  class RawData < Data
    def self.regex
      /\A--data-raw\Z/
    end

    def options
      { raw: true }
    end
  end

  class URLEncodeData < Data
    def self.regex
      /\A--data-urlencode\Z/
    end

    def options
      { url_encode: true }
    end
  end

  class URL < Base
    def self.===(word)
      word =~ URI::DEFAULT_PARSER.make_regexp
    end

    def parse(word)
      request.url = word
    end
  end

  class NonStrictURL < Base
    def self.regex
      /\./
    end

    def parse(word)
      request.url = "http://#{word}" if request.url == ''
    end
  end
end

Here's the final code we end up with:

module Word
  def self.parse(request, words, word)
    subclass = Word::Base.subclasses
                         .detect { |klass| klass === word }
    subclass&.new(words, request)&.parse(word)
  end

  class Base
    def initialize(words, request)
      @words = words
      @request = request
    end

    # List of subclasses, first-inherited class appears first.
    def self.subclasses
      ObjectSpace.each_object(Base.singleton_class)
                 .reject { |klass| klass == Base }.reverse
    end

    # Used to check if the word given matches the subclass' regex.
    def self.===(word)
      word =~ regex
    end

    def self.regex
      raise NotImplementedError,
            "#{name} doesn't have .regex defined"
    end

    def parse(*)
      raise NotImplementedError,
            "#{self.class.name} doesn't have #parse defined"
    end

    private

    attr_reader :words
    attr_reader :request
  end

  class Header < Base
    def self.regex
      /\A(-H|--header)\Z/
    end

    def parse(*)
      header = extract_header(words.shift)

      return unless header

      request.headers[header.keys.first] = header.values.first
    end

    private

    def extract_header(header)
      [header.split(/: (.+)/)].to_h
    rescue StandardError
      nil
    end
  end

  class Compressed < Base
    def self.regex
      /\A--compressed\Z/
    end

    def parse(*)
      request.headers['Accept-Encoding'] ||= 'deflate, gzip'
    end
  end

  class Request < Base
    def self.regex
      /\A-X|\A--request\Z/
    end

    def parse(word)
      request.method = extract_method(word)
    end

    private

    # Needed for extracting -XPUT and similar formats.
    def extract_method(word)
      return words.shift unless word =~ /^-X/

      if !(method = word.match(/^-X(.*)/)[1]).empty?
        method
      else
        words.shift
      end
    end
  end

  class Authorization < Base
    def self.regex
      /\A(-u|--user)\Z/
    end

    def parse(*)
      request.headers['Authorization'] ||=
        "Basic #{Base64.strict_encode64(words.shift)}"
    end
  end

  class Cookie < Base
    def self.regex
      /\A(-b|--cookie)\Z/
    end

    def parse(*)
      request.headers['Set-Cookie'] ||= words.shift
    end
  end

  class UserAgent < Base
    def self.regex
      /\A(-A|--user-agent)\Z/
    end

    def parse(*)
      request.headers['User-Agent'] ||= words.shift
    end
  end

  class Head < Base
    def self.regex
      /\A(-I|--head)\Z/
    end

    def parse(*)
      request.method = 'HEAD'
    end
  end

  class Data < Base
    def self.regex
      /\A(-d|--data|--data-ascii)\Z/
    end

    def options
      {}
    end

    def parse(*)
      request.method = 'POST' if request.method == 'GET' ||
                                 request.method == 'HEAD'
      request.headers['Content-Type'] ||=
        'application/x-www-form-urlencoded'

      data = extract_data(options)

      request.body = if request.body.empty?
                       data
                     else
                       "#{request.body}&#{data}"
                     end
    end

    def extract_data(args)
      data = words.shift
      data = read_file(data, args) if data =~ /\[email protected]/ ||
                                      args[:raw]
      data
    end

    def read_file(word,
                  strip_newlines: true,
                  raw: false,
                  url_encode: false)
      path = raw ? word : word[1..-1]
      content = File.read(path)
      content.gsub!(/\n|\r/, '') if strip_newlines
      content = CGI.escape(content) if url_encode
      content
    rescue Errno::ENOENT
      file_path = File.expand_path(data[1..-1])
      raise FileNotFoundError,
            "Cannot read file: #{file_path}. Does it exist?"
    end
  end

  class StripNewlinesData < Data
    def self.regex
      /\A--data-binary\Z/
    end

    def options
      { strip_newlines: false }
    end
  end

  class RawData < Data
    def self.regex
      /\A--data-raw\Z/
    end

    def options
      { raw: true }
    end
  end

  class URLEncodeData < Data
    def self.regex
      /\A--data-urlencode\Z/
    end

    def options
      { url_encode: true }
    end
  end

  class URL < Base
    def self.===(word)
      word =~ URI::DEFAULT_PARSER.make_regexp
    end

    def parse(word)
      request.url = word
    end
  end

  class NonStrictURL < Base
    def self.regex
      /\./
    end

    def parse(word)
      request.url = "http://#{word}" if request.url == ''
    end
  end
end

class Parser
  def initialize(request)
    @request = request
  end

  def parse(words)
    self.words = words

    while (word = words.shift)
      parse_word(word)
    end
  end

  private

  attr_accessor :words
  attr_reader :request

  def parse_word(word)
    Word.parse(request, words, word)
  end
end

module ParseCurl
  class << self
    def parse(command)
      self.request = Request.new
      self.parser = Parser.new(request)

      parser.parse(Shellwords.split(command.strip))

      request.to_h
    end

    private

    attr_accessor :request
    attr_accessor :parser
  end
end

ParseCurl.parse(ARGV.first)

Note how each class contains knowledge specific to the argument we're trying to cover with it and nothing else. I think that's an improvement since it's still easy to support new arguments, and we don't pollute a single class with many methods.