Ruby vs Python: Blowfish ECB encryption

Hi,

I'm trying to understand why there is a difference between Ruby and
Python when using Blowfish encryption in ECB mode. Using the same
algorithm parameters, and the same input, the encrypted output is
completely different.

Behold these two code samples:

RUBY==================
require 'digest'
require 'openssl'

cleartext = 'whatevs'

md5 = Digest::MD5.new
md5.update(cleartext)

payload = (md5.digest + cleartext).ljust(0x80, "\x00")

puts "plaintext: " + payload.unpack('H*')[0]

secret_key = "supersecretsupersecret"

cipher = OpenSSL::Cipher::Cipher.new("bf-ecb").send :encrypt
cipher.key = secret_key
cipher.padding = 0
binary_data = (cipher.update(payload) << cipher.final)

puts "ciphertext: " + binary_data.unpack('H*')[0]

cipher = OpenSSL::Cipher::Cipher.new("bf-ecb").send :decrypt
cipher.key = secret_key
cipher.padding = 0
str = (cipher.update(binary_data) << cipher.final)

puts "decrypted: " + str.unpack('H*')[0]
RUBY==================

PYTHON================
from Crypto.Cipher import Blowfish
from Crypto.Hash import MD5
import binascii

cleartext = 'whatevs'
md5_key = MD5.new(cleartext).digest()
payload = (md5_key + cleartext).ljust(0x80, "\x00")
print "plaintext: " + binascii.hexlify(payload)
secret_key = "supersecretsupersecret"

ciphertext = Blowfish.new(secret_key, Blowfish.MODE_ECB).encrypt(payload)

print "ciphertext: " + binascii.hexlify(ciphertext)

print "decrypted: " + binascii.hexlify(Blowfish.new(secret_key,
1).decrypt(ciphertext))
PYTHON================

Running these, I get the following:
~ > ruby /tmp/test.rb
plaintext:
030fa889aa00fc6554023a9aad8c9ca177686174657673000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
ciphertext:
e7fd839dbc4762226b1086170e8227c2e2e4c54ee4f51fbd2e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc45
decrypted:
030fa889aa00fc6554023a9aad8c9ca177686174657673000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

~ > python /tmp/test.py
plaintext:
030fa889aa00fc6554023a9aad8c9ca177686174657673000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
ciphertext:
992f5dad8650fb447416693d8d1f07e869c63d8798234e933bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee
decrypted:
030fa889aa00fc6554023a9aad8c9ca177686174657673000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

As you can see, the plaintext and decrypted ciphertext are the same, but
the ciphertexts are different.

Why is this happening? This is a major pain as it means I cannot interop
with Python. There are other gems for Blowfish encryption, but none of
them have ECB mode (which is the one I'm interested in - and yes, before
you say it, I know ECB is insecure).

Am I doing something wrong? Or is this a deeper problem in Python or Ruby?

Regards,
Pedro

You are probably running into this problem:

···

On Wed, Nov 30, 2016 at 1:19 PM, Pedro Ribeiro <pedrib@gmail.com> wrote:

Hi,

I'm trying to understand why there is a difference between Ruby and
Python when using Blowfish encryption in ECB mode. Using the same
algorithm parameters, and the same input, the encrypted output is
completely different.

Behold these two code samples:

RUBY==================
require 'digest'
require 'openssl'

cleartext = 'whatevs'

md5 = Digest::MD5.new
md5.update(cleartext)

payload = (md5.digest + cleartext).ljust(0x80, "\x00")

puts "plaintext: " + payload.unpack('H*')[0]

secret_key = "supersecretsupersecret"

cipher = OpenSSL::Cipher::Cipher.new("bf-ecb").send :encrypt
cipher.key = secret_key
cipher.padding = 0
binary_data = (cipher.update(payload) << cipher.final)

puts "ciphertext: " + binary_data.unpack('H*')[0]

cipher = OpenSSL::Cipher::Cipher.new("bf-ecb").send :decrypt
cipher.key = secret_key
cipher.padding = 0
str = (cipher.update(binary_data) << cipher.final)

puts "decrypted: " + str.unpack('H*')[0]
RUBY==================

PYTHON================
from Crypto.Cipher import Blowfish
from Crypto.Hash import MD5
import binascii

cleartext = 'whatevs'
md5_key = MD5.new(cleartext).digest()
payload = (md5_key + cleartext).ljust(0x80, "\x00")
print "plaintext: " + binascii.hexlify(payload)
secret_key = "supersecretsupersecret"

ciphertext = Blowfish.new(secret_key, Blowfish.MODE_ECB).encrypt(payload)

print "ciphertext: " + binascii.hexlify(ciphertext)

print "decrypted: " + binascii.hexlify(Blowfish.new(secret_key,
1).decrypt(ciphertext))
PYTHON================

Running these, I get the following:
~ > ruby /tmp/test.rb
plaintext:
030fa889aa00fc6554023a9aad8c9ca17768617465767300000000000000
000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000
0000000000000000
ciphertext:
e7fd839dbc4762226b1086170e8227c2e2e4c54ee4f51fbd2e4bd6de84ea
fc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de
84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4b
d6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc45
2e4bd6de84eafc45
decrypted:
030fa889aa00fc6554023a9aad8c9ca17768617465767300000000000000
000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000
0000000000000000

~ > python /tmp/test.py
plaintext:
030fa889aa00fc6554023a9aad8c9ca17768617465767300000000000000
000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000
0000000000000000
ciphertext:
992f5dad8650fb447416693d8d1f07e869c63d8798234e933bb878160d77
5aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb87816
0d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb8
78160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee
3bb878160d775aee
decrypted:
030fa889aa00fc6554023a9aad8c9ca17768617465767300000000000000
000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000
0000000000000000

As you can see, the plaintext and decrypted ciphertext are the same, but
the ciphertexts are different.

Why is this happening? This is a major pain as it means I cannot interop
with Python. There are other gems for Blowfish encryption, but none of
them have ECB mode (which is the one I'm interested in - and yes, before
you say it, I know ECB is insecure).

Am I doing something wrong? Or is this a deeper problem in Python or Ruby?

Regards,
Pedro

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

--
Tony Arcieri

Hi Tony,

I don't that's the case... if you look closely you will see I'm setting
the key after calling encrypt. In any case, I've tried swapping them
(setting the key first, then calling encrypt) and I actually get the
same result.

Regards,
Pedro

···

On 30/11/16 21:25, Tony Arcieri wrote:

You are probably running into this problem:

OpenSSL::Cipher::Cipher produces garbage result if #encrypt is called after setting key · Issue #73 · ruby/openssl · GitHub

On Wed, Nov 30, 2016 at 1:19 PM, Pedro Ribeiro <pedrib@gmail.com > <mailto:pedrib@gmail.com>> wrote:

    Hi,

    I'm trying to understand why there is a difference between Ruby and
    Python when using Blowfish encryption in ECB mode. Using the same
    algorithm parameters, and the same input, the encrypted output is
    completely different.

    Behold these two code samples:

    RUBY==================
    require 'digest'
    require 'openssl'

    cleartext = 'whatevs'

    md5 = Digest::MD5.new
    md5.update(cleartext)

    payload = (md5.digest + cleartext).ljust(0x80, "\x00")

    puts "plaintext: " + payload.unpack('H*')[0]

    secret_key = "supersecretsupersecret"

    cipher = OpenSSL::Cipher::Cipher.new("bf-ecb").send :encrypt
    cipher.key = secret_key
    cipher.padding = 0
    binary_data = (cipher.update(payload) << cipher.final)

    puts "ciphertext: " + binary_data.unpack('H*')[0]

    cipher = OpenSSL::Cipher::Cipher.new("bf-ecb").send :decrypt
    cipher.key = secret_key
    cipher.padding = 0
    str = (cipher.update(binary_data) << cipher.final)

    puts "decrypted: " + str.unpack('H*')[0]
    RUBY==================

    PYTHON================
    from Crypto.Cipher import Blowfish
    from Crypto.Hash import MD5
    import binascii

    cleartext = 'whatevs'
    md5_key = MD5.new(cleartext).digest()
    payload = (md5_key + cleartext).ljust(0x80, "\x00")
    print "plaintext: " + binascii.hexlify(payload)
    secret_key = "supersecretsupersecret"

    ciphertext = Blowfish.new(secret_key,
    Blowfish.MODE_ECB).encrypt(payload)

    print "ciphertext: " + binascii.hexlify(ciphertext)

    print "decrypted: " + binascii.hexlify(Blowfish.new(secret_key,
    1).decrypt(ciphertext))
    PYTHON================

    Running these, I get the following:
    ~ > ruby /tmp/test.rb
    plaintext:
    030fa889aa00fc6554023a9aad8c9ca177686174657673000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
    ciphertext:
    e7fd839dbc4762226b1086170e8227c2e2e4c54ee4f51fbd2e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc452e4bd6de84eafc45
    decrypted:
    030fa889aa00fc6554023a9aad8c9ca177686174657673000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

    ~ > python /tmp/test.py
    plaintext:
    030fa889aa00fc6554023a9aad8c9ca177686174657673000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
    ciphertext:
    992f5dad8650fb447416693d8d1f07e869c63d8798234e933bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee3bb878160d775aee
    decrypted:
    030fa889aa00fc6554023a9aad8c9ca177686174657673000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

    As you can see, the plaintext and decrypted ciphertext are the same, but
    the ciphertexts are different.

    Why is this happening? This is a major pain as it means I cannot interop
    with Python. There are other gems for Blowfish encryption, but none of
    them have ECB mode (which is the one I'm interested in - and yes, before
    you say it, I know ECB is insecure).

    Am I doing something wrong? Or is this a deeper problem in Python or
    Ruby?

    Regards,
    Pedro

    Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org
    <mailto:ruby-talk-request@ruby-lang.org>?subject=unsubscribe>
    <http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk
    <http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;&gt;

--
Tony Arcieri

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

try,
1 Using padding in encryption
2 AES Python encryption and Ruby encryption - different behaviour? - Stack Overflow

best regards
--botp

···

On Thu, Dec 1, 2016 at 5:19 AM, Pedro Ribeiro <pedrib@gmail.com> wrote:

I'm trying to understand why there is a difference between Ruby and
Python when using Blowfish encryption in ECB mode. Using the same
algorithm parameters, and the same input, the encrypted output is
completely different.

Oh, sorry, I glanced over the "send :encrypt" (BTW that's a rather odd way
of doing that... I guess your eventual intent was to parameterize encrypt
vs decrypt?)

···

On Wed, Nov 30, 2016 at 1:31 PM, Pedro Ribeiro <pedrib@gmail.com> wrote:

I don't that's the case... if you look closely you will see I'm setting
the key after calling encrypt

--
Tony Arcieri

To be honest, I just did some copy paste of that code.
However, even if I change it and put it in a more readable way, I still
get the same results.

I'm starting to think I'll have to dig around in the Ruby and Python
code, and possibly port the Python code to Ruby.

Before I do that, are there any enlightened souls that can explain this
discrepancy?

Regards,
Pedro

···

On 30/11/16 22:05, Tony Arcieri wrote:

On Wed, Nov 30, 2016 at 1:31 PM, Pedro Ribeiro <pedrib@gmail.com > <mailto:pedrib@gmail.com>> wrote:

    I don't that's the case... if you look closely you will see I'm setting
    the key after calling encrypt

Oh, sorry, I glanced over the "send :encrypt" (BTW that's a rather odd
way of doing that... I guess your eventual intent was to parameterize
encrypt vs decrypt?)

--
Tony Arcieri

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Thanks for the help botp. Your answer put me in the right track.
I went to the Blowfish vector page and tried different vectors to
understand what is wrong.

It seems that the Ruby OpenSSL makes some assumptions with regards to
key length - I'm not sure if this is a bug or not. The fix for the code
I pasted before was very simple: when initializing the cipher, I have to
specify the key_len before setting the key:

cipher = OpenSSL::Cipher::Cipher.new("bf-ecb").send :encrypt
cipher.key_len = secret_key.length
cipher.key = secret_key
cipher.padding = 0
binary_data = (cipher.update(payload) << cipher.final)

This makes it work correctly and match the Python output.

Not sure if this is a bug or not, if you think it is I'll just create a
ticket in the tracker.

Regards,
Pedro

···

On 02/12/16 10:00, botp wrote:

On Thu, Dec 1, 2016 at 5:19 AM, Pedro Ribeiro <pedrib@gmail.com> wrote:

I'm trying to understand why there is a difference between Ruby and
Python when using Blowfish encryption in ECB mode. Using the same
algorithm parameters, and the same input, the encrypted output is
completely different.

try,
1 Using padding in encryption
2 AES Python encryption and Ruby encryption - different behaviour? - Stack Overflow

best regards
--botp

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

imho, ruby follows standard behaviour by padding (padding=1),
[python does not follow standard]
just because ruby's result differs from that of python, doesnt mean
that ruby is wrong, no? anyway, you can ask the core team, they are
always happy to help.
see eg Using padding in encryption

kind regards --botp

···

On Fri, Dec 2, 2016 at 7:32 PM, Pedro Ribeiro <pedrib@gmail.com> wrote:

cipher.padding = 0
...
This makes it work correctly and match the Python output.
Not sure if this is a bug or not, if you think it is I'll just create a
ticket in the tracker.