organized flames

Search

ActiveResource and Shallow Nested Routes

Posted on November 04, 2009 by Michael

I recently tried to dive into ActiveResource, which allows a Ruby (usually Ruby on Rails) application to connect to a remote RESTful API and treat it almost as if it were local. There were some problems with my API from ActiveResource’s point of view.

Firstly, I use nested routes. This isn’t really a problem as it is possible to specify a prefix to add to the path, and even to substitution on this path. However, Rails added the concept of a “shallow nested route” a while back.

A shallow nested route is a short-hand notation which allows one to appear to scope out collections (that is, a list of things) from the actual thing itself. For example, in my application (https://dlv.isc.org/) I have:

/users/123 <– specific userid

/users/123/zones <— list of all zones for user 123

/zones/456 <— specific zone

This allows me to look at a specific user’s zones (or them, their own zones) without having to do some sort of special back-end processing which relies on something not in the path, such as the @current_user instance variable. After all, how would an admin list a user’s zones if /zones returned the current user’s zones only? Through an admin interface probably, but that seems messy. Shallow routes are seemingly more clean.

However, they break ActiveResource’s concept of the world.

With a little trickery, however, I managed to get shallow nested routes to work without having to do much additional work. I did have to pass in an additional argument to the collection list(s) however.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class Zone < ActiveResource::Base
  self.site = SITE
  self.user = USERNAME
  self.password = PASSWORD
  
  def self.collection_path(prefix_options = {}, query_options = nil)
    prefix_options, query_options = split_options(prefix_options) if query_options.nil?
    "/users/#{USERID}#{prefix(prefix_options)}#{collection_name}.#{format.extension}#{query_string(query_options)}"
  end
end

class Dnskey < ActiveResource::Base
  self.site = SITE
  self.user = USERNAME
  self.password = PASSWORD
  
  def self.collection_path(prefix_options = {}, query_options = nil)
    prefix_options, query_options = split_options(prefix_options) if query_options.nil?
    z = ''
    if query_options.has_key?(:zone_id)
      z = "/zones/#{query_options[:zone_id]}"
      query_options.delete(:zone_id)
    end
    "#{z}#{prefix(prefix_options)}#{collection_name}.#{format.extension}#{query_string(query_options)}"
  end
end

In my case, I used a constant USERID to the Zone’s collection, but left the specific item (aka “elementpath”) alone. For Dnskeys, where I needed to pass in different zoneid values, I had to do a small trick.

I call this as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
all_zones = Zones.find :all # returns a list of all zones for the USERID
z = Zone.find(1) # returns only Zone with the id of 1
z.destroy # destroys the zone, uses the element_path()

z = Zone.find(2)
keys = Dnskey.find(:all, { :zone_id => z.id }) # disgusting, but works

#
# This is a hack.  I want to use Dnskey.create here, but it won't work since I cannot
# pass the zone_id along, and there are no association hints so I can't use
# zone.dnskeys.create() like I can with ActiveRecord.
#
d = Dnskey.new(:flags => key.flags, :algorithm => key.algorithm.to_i, ...)
d.prefix_options[:zone_id] = zone.id
d.save

Ruby and OpenSSL

Posted on February 28, 2009 by Michael

I recently had to do some DNSSEC-type (somewhat low-level) cryptography work, and found the seeming lack of Ruby OpenSSL documentation a big pain. I found numerous examples of how OpenSSL is commonly used with PEM-encoded keys, but precious little information on low-level key loading. To save others the trouble of having to dig up some of this, I’ve collected some short examples of how to do low-level RSA and DSA building from a lower level than most use.

This table summarizes the variables which need to be set to use an RSA public and private key.

RSA Keys

Key TypeItemDescription
RSA PublicePublic Exponent
RSAnModulus
RSA PrivatedPrivate Exponent
RSA PrivatepPrime 1
RSA PrivateqPrime 2
RSA Privatedmq1Exponent 1
RSA Privatedmp1Exponent 2
RSA PrivateiqmpCoefficient

Thus, in order to make a working RSA public key (so the method key.publicencrypt() or key.publicdecrypt() work) you must set at least n and e. For a working private key, you would need to load all of the items. Exposing any of the items marked as “RSA Private” above will cause a key compromise.

RSA Example

In this example, a 128-bit RSA key is loaded from numerical values. In DNSSEC, the public key is stored in the DNSKEY record for the zones. Don’t use these numbers for real crypto; the short key length is used only to make the numbers short enough to fit in the screen width. For real work, 1024 is probably a reasonable minimum length for short-lived uses, and 2048 for longer-term use.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
require 'openssl'

#
# Build a RSA public key.  We only need to load two things
# here in order to use the public key to use it to encrypt,
# sign, or verify.
#
pub = OpenSSL::PKey::RSA::new
pub.e = 65537
pub.n = 216457604585180710748301099018726389113

# At this point, this will work:
crypted = pub.public_encrypt("test")

#
# Build an RSA private key.  For the private key to work, we need
# to load the entire key, private and public components.  As we
# should have access to both, this is not really a problem.
#
prv = OpenSSL::PKey::RSA::new
prv.e = 65537
prv.d = 178210827022942698143906513631075003381
prv.n = 216457604585180710748301099018726389113
prv.p = 15294921647876231099
prv.q = 14152253249053866587
prv.dmp1 = 6806715058393856237
prv.dmq1 = 637679537428568107
prv.iqmp = 6672106206837437412

# Now we have a working private key.
puts prv.private_decrypt(crypted) # prints "test"

DSA keys

A DSA key is more or less the same, just with different variable names. It is also split into a public and private part, and the key can be loaded from individual components just as easily.

Key TypeItemDescription
DSA Publicpub_keyPublic Key
DSAqPrime 1
DSApPrime 2
DSAgMultiplicative order modulo p is q
DSA Privatepriv_keyPrivate Key

DSA Example

Unfortunately, this example has some numbers which are too long to display nicely. I have used a trick to convert them from strings into integers so they will fit here. Normally you would not need to do this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
require 'openssl'

#
# Build an DSA private key.  For the private key to work, we need
# to load the entire key, private and public components.  As we
# should have access to both, this is not really a problem.
#
prv = OpenSSL::PKey::DSA::new
prv.pub_key = ("899167044393666062859565588228279347268072456516837337" +
               "963353916587148226144760114643916732975837345856985656" +
               "3340384802383806137452386519280693373122367959").to_i
prv.p = ("952649509730281181203079535805855260554748337655197352471196" +
         "869232197576949258404031665397657842790773780623545384978542" +
         "6685417827665656974405272756289291").to_i
prv.q = 903197981571669745498020976355730183999507610553
prv.g = ("535694721480531756072717909769318961974692885092552247120424" +
         "749877864650255208980198391972633196543370921493242375015765" +
         "755160911031468160738717891191998").to_i
prv.priv_key = 557886499717422048101097620625259920363848888840

# At this point, this will work:
signature = prv.sign(OpenSSL::Digest::DSS1.new, "test")

#
# Build a DSA public key.  We only need to load two things
# here in order to use the public key to use it to encrypt,
# sign, or verify.
#
pub = OpenSSL::PKey::DSA::new
pub.pub_key = ("899167044393666062859565588228279347268072456516837337" +
               "963353916587148226144760114643916732975837345856985656" +
               "3340384802383806137452386519280693373122367959").to_i
pub.p = ("952649509730281181203079535805855260554748337655197352471196" +
         "869232197576949258404031665397657842790773780623545384978542" +
         "6685417827665656974405272756289291").to_i
pub.q = 903197981571669745498020976355730183999507610553
pub.g = ("535694721480531756072717909769318961974692885092552247120424" +
         "749877864650255208980198391972633196543370921493242375015765" +
         "755160911031468160738717891191998").to_i

# Now we have a working private key.  Verify the signature
if pub.verify(OpenSSL::Digest::DSS1.new, signature, "test")
  puts "Signature verified."
else
  puts "Signature verification failed."
end

Find conditions from params[]

Posted on February 28, 2009 by Michael

I have often wished to add a trivial search capability to my controllers which would allow searching on different fields. Without getting into a mess of many different if statements each of which has a different set of conditions, I use something much like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def index
  cond_hash = {}
  cond_strings = []

  if params[:search]
    cond_hash[:login] = "%#{params[:search]}%"
    cond_hash[:email] = "%#{params[:search]}%"
    cond_strings << "(login ILIKE :login OR email ILIKE :email)"
  end
  if params[:search_address]
    cond_hash[:address] = params[:search_address]
    cond_strings << "(address == :address)"
  end

  conditions = cond_strings.join(" AND ")
  @users = User.all :conditions => [ conditions, cond_hash ]
end

It may be safer to use users.login, users.email, and users.address in the SQL-like strings above.

Ruby Regular Expression Gotchas

Posted on February 26, 2009 by Michael

I love Ruby. I love Ruby on Rails. Rarely have I found a language or a framework that just works.

However, you still have to know the finer details sometimes. I recently made a model for a DNS zone. The name in the model is the “front part” of a fully qualified domain name. For instance, if zone.name = “foo” then I would write the name into my name server’s configuration files as “foo.example.com.”

Knowing that people were evil, I saw that if a user put a string in like “example.com. NS hackerz-will-someday-rule-the-earth.ru.\nfoo” I would happily write out two strings, one being rather bad.

Knowing how easy this sort of data validation is in Rails, I made my model look like:

1
2
3
4
5
6
7
class Zone < ActiveRecord::Base
  validates_presence_of :name
  validates_uniqueness_of :name
  validates_format_of :name,
    :with => /^[a-zA-Z0-9\-\_\.]+$/,
    :message => "contains invalid characters."
end

Happy, I ran a few tests using my browser and found that I could not insert names with spaces, colons, tabs, etc. Then, several days later, I decided it was time to write tests for this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
require 'test_helper'
class ZoneTest < ActiveSupport::TestCase
  def test_name_with_newline_fails
    z = Zone.new(:name => "test\nzone")
    assert !z.valid?
    assert z.errors.on(:name)
  end

  def test_name_with_space_fails
    z = Zone.new(:name => "test zone")
    assert !z.valid?
    assert z.errors.on(:name)
  end
end

Imagine my surprise when testnamewithspacefails() passed, and the one I was most worried about, testnamewithnewlinefails(), did not!

Not all regular expressions are alike.

The problem is in what I thought ^ and $ actually matched. I thought these meant “match the beginning and ending of the string.” However, it turns out it means “match the beginning and ending of each line contained in the string,” where lines are divided by newlines. Ooops.

Changing ^ into \A and $ into \Z fixed this problem. Now I’m auditing all the code in this application to see if there are other problems like this.

This is just one thing to add to an ever-growing security checklist for my Rails work. It’s also a very typical security hole: programmer error.

String#bitruncate

Posted on February 21, 2009 by Michael

And that’s ‘“bi-truncate” not “bit-uncate”.

What’s it do?

1
2
"This is a test".bitruncate(:length => 6) ==> "Thi...est"
"This is a test".bitruncate(:elength => 6) ==> "...a test"

The default options are { :length => 30 } which will produce 15 characters from the front and 15 from the end, putting … marks in the middle.

For rails, I put this in my lib/core_extensions.rb file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
#
# Copyright (c) 2009 Michael Graff.  All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above
#    copyright notice, this list of conditions and the following
#    disclaimer in the documentation and/or other materials provided
#    with the distribution.
# 3. The name of Michael Graff may not be used to endorse or promote
#    products derived from this software without specific prior
#    written permission.
#
# THIS SOFTWARE IS PROVIDED BY Michael Graff ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
# PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL Micahel Graff
# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
# OF SUCH DAMAGE.
#

class String
  #
  # Truncate from both ends of a string.  The :length parameter, which defaults
  # to 30, will return the first 15 and the last 15 characters from a string
  # if it is longer than 30 characters.  If it is shorter, the entire string
  # is returned.
  #
  # Another way to specify the front and back portions are with :flength and
  # :elength.  If you specify one of these but not the other then you will
  # not get the missing part.  e.g., :flength => 10 alone will return only
  # the first 10 charcters of the string.  This is the same as the standard
  # truncate(s, :length => 10) helper.
  #
  # If a :length parameter is provided it will override any other lengths
  # specified.
  #
  def bitruncate(options = {})
    maxlength = options[:length] || 0
    flength = options[:flength] || 0
    elength = options[:elength] || 0
    omission = options[:omission] || '...'

    if maxlength == 0 && flength == 0 && elength == 0
      maxlength = 30
    end
    if maxlength != 0
      flength = maxlength / 2
      elength = maxlength / 2
    end

    return self if length <= (flength + elength)

    front = ''
    back = ''
    if flength > 0
      front = self[0..(flength - 1)]
    end
    if elength > 0
      back = self[(length - elength)..(length)]
    end

    front + omission + back
  end
end

Checking Credit Card Numbers in Ruby

Posted on March 24, 2008 by Michael

This is not meant to be an exhaustive list of all possible numbers, nor the only or best method to verify that they pass the “checksum” test, but here’s what I came up with.

I wrote this mostly to link a Ruby version of the code to Wikipedia’s article on Luhn checksum validation, since nearly every other language in use was listed, but Ruby was sadly missing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
#!/usr/bin/env ruby

#
# Copyright (c) 2008 Michael Graff.  All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above
#    copyright notice, this list of conditions and the following
#    disclaimer in the documentation and/or other materials provided
#    with the distribution.
# 3. The name of Michael Graff may not be used to endorse or promote
#    products derived from this software without specific prior
#    written permission.
#
# THIS SOFTWARE IS PROVIDED BY Michael Graff ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
# PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL Micahel Graff
# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
# OF SUCH DAMAGE.
#


class Luhn
  public
  def self.check_luhn(s)
    s.gsub!(/[^0-9]/, "")
    ss = s.reverse.split(//)

    alternate = false
    total = 0
    ss.each do |c|
      if alternate
        total += double_it(c.to_i)
      else
        total += c.to_i
      end
      alternate = !alternate
    end
    (total % 10) == 0
  end

  private
  def self.double_it(i)
    i = i * 2
    if i > 9
      i = i % 10 + 1
    end
    i
  end

end

if $0 == __FILE__
  def test_valid(s)
    result = Luhn::check_luhn(s)
    if result
      puts "VALID: #{s}"
    else
      puts "INVALID: #{s} (should be valid)"
    end
  end

  test_valid('5105 1051 0510 5100') # Mastercard
  test_valid('5555 5555 5555 4444') # Mastercard

  test_valid('4222 2222 2222 2')    # Visa
  test_valid('4111 1111 1111 1111') # Visa
  test_valid('4012 8888 8888 1881') # Visa

  test_valid('3782 8224 6310 005')  # American Express
  test_valid('3714 4963 5398 431')  # American Express
  test_valid('3787 3449 3671 000')  # American Express Corporate
  test_valid('3782 8224 6310 005')  # Amex
  test_valid('3400 0000 0000 009')  # Amex
  test_valid('3700 0000 0000 002')  # Amex

  test_valid('38520000023237')      # Diners Club (14 digits)
  test_valid('30569309025904')      # Diners Club (14 digits)

  test_valid('6011111111111117')    # Discover (16 digits)
  test_valid('6011 0000 0000 0004') # Discover
  test_valid('6011 0000 0000 0012') # Discover
  test_valid('6011000990139424')    # Discover (16 digits)
  test_valid('6011601160116611')    # Discover (16 digits)

  test_valid('3530111333300000')    # JCB (16 digits)
  test_valid('3566002020360505')    # JCB (16 digits)

  test_valid('5431111111111111')    # Mastercard (16 digits)
end

RailsMode (not ENV['RAILS_ENV'])

Posted on October 27, 2007 by Michael

RailsMode

I’ve been spreading things like this all around my code, where I wanted to do something differently in production vs. development mode. Previously, I’d write something like this:

1
2
3
if ENV['RAILS_ENV'] == 'production'
  ... perform magic ...
end

While this is pretty simple, it just didn’t feel very DRY. So, I decided to use this as a reason to learn about modules and mixins.

I have a generic plug-in in vendor/plugins that I put small bits of code like this. You might as well, but if you don’t, you can drop this in a helper.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
module RailsMode
  def railsmode(*list)
    list.map! do |item|
      item.to_s
    end

    if block_given?
      if list.include?(ENV['RAILS_ENV'])
        yield
      end
    else
      return list.include?(ENV['RAILS_ENV'])
    end
  end
end

I also put this line in my environment.rb file:


include RailsMode

This mixes the module into the current class. Doing this in environment.rb makes it available everywhere in rails. Putting that line in a specific file would also work, such as a controller, or a helper.

With this, I can now write:

1
2
3
if railsmode(:production)
  ... perform magic ...
end

I can also check for multiple modes at once:

1
2
3
if railsmode(:production, :development)
  ... perform magic ...
end

And of course, who needs an if when I can pass in a block:

1
2
3
railsmode(:production) do
  ... perform magic ...
end