Securing 'pickle'

Re: Securing 'pickle'

On Fri, 11 Jul 2003 13:20:48 +1200, David McNab wrote:[color=blue]
> I'm writing a web app framework which stores pickles in client
> cookies.[/color]

Sounds like a waste of bandwidth, in addition to the security concerns
you raise.

Why not store the pickles on the server, and set a session cookie to
refer to them? That way, you only send a short session ID instead of
the whole pickle, and messing with the cookie doesn't alter the pickles.

(Mmm, all this talk of food...)

--
\ "I don't know anything about music. In my line you don't have |
`\ to." -- Elvis Aaron Presley (1935-1977) |
_o__) |
http://bignose.squidly.org/ 9CFE12B0 791A4267 887F520C B7AC2E51 BD41714B

Re: Securing 'pickle'

Paul Rubin wrote:
[color=blue]
> Because now you need a mechanism to store the session info on the
> server, and you might want it to work across multiple load-balanced
> servers that fail over to one another, etc.[/color]

That's far superior to presenting the opportunity to exploits in the
first place, in my opinion. Depending on the contents of the contents
of that cookie, what you suggest may not be a problem at all (depending
on how critical the data contained therein is).

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ I would rather understand one cause than be king of Persia.
\__/ Democritus

Re: Securing 'pickle'

Erik Max Francis <max@alcyone.co m> writes:[color=blue][color=green]
> > Because now you need a mechanism to store the session info on the
> > server, and you might want it to work across multiple load-balanced
> > servers that fail over to one another, etc.[/color]
>
> That's far superior to presenting the opportunity to exploits in the
> first place, in my opinion. Depending on the contents of the contents
> of that cookie, what you suggest may not be a problem at all (depending
> on how critical the data contained therein is).[/color]

I'm not sure what you're saying here. My suggestion is to
authenticate the cookies with a cryptographic checksum and verify the
authentication before deserializing the cookies. That's probably the
simplest approach. Keeping session info on a multi-process server (or
worse, a multi-server network) needs some kind of concurrent storage
mechanism. I don't see a robust, secure, low-overhead way to do that
with out-of-the-box Python. Any suggestions?

Re: Securing 'pickle'

On Thu, 2003-07-10 at 20:20, David McNab wrote:[color=blue]
> I'm writing a web app framework which stores pickles in client cookies.
>
> The obvious security risk is that some 5cr1p7 X1ddi35 will inevitably try
> tampering with the cookie and malforming it in an attempt to get the
> server-side python code to run arbitrary code, or something similarly
> undesirable.
>
> To protect against this, I've subclassed pickle.Unpickle r, and added
> overrides of the methods load_global, load_inst, load_obj and find_class.[/color]

A much easier way to secure your pickle is to sign it, like:

cookie = dumps(object)
secret = 'really secret!'
hasher = md5.new()
hasher.update(s ecret)
hasher.update(c ookie)
cookie_signatur e = md5.digest()

You may then wish to base64 encode both (.encode('base6 4')), pop them
into one value, and you're off. Though I suppose at that point you may
be hitting the maximum value of a cookie. Hidden fields will work
nicely, though.

Decoding and verifying is an exercise left to the reader.

Ian

Re: Securing 'pickle'

On Thu, 2003-07-10 at 20:52, Erik Max Francis wrote:[color=blue][color=green]
> > Because now you need a mechanism to store the session info on the
> > server, and you might want it to work across multiple load-balanced
> > servers that fail over to one another, etc.[/color]
>
> That's far superior to presenting the opportunity to exploits in the
> first place, in my opinion. Depending on the contents of the contents
> of that cookie, what you suggest may not be a problem at all (depending
> on how critical the data contained therein is).[/color]

Security isn't a big deal -- or rather, securing cookies isn't a big
deal. I think reliability will be a bigger problem. Cookies can cause
problems even when you are just storing a simple session ID. If you
start storing more information you're likely to run up against other
problems -- cookies can be hard to dispose of, who knows where they'll
get chopped off to preserve storage (it happens quickly), and IE has a
bug where you can't redirect and set a cookie at the same time, which
can really drive you crazy if you don't know about it.

Hidden fields are a much better way of keeping information on the
client. They tend to make for more navigable pages too. But if you
really want session, not transaction data, then you just need to figure
out server-side sessions. The biggest advantage of a web application is
that it runs in a controlled environment (the server) and you should
take advantage of that.

Ian

Re: Securing 'pickle'

Ian Bicking <ianb@colorstud y.com> writes:[color=blue]
> A much easier way to secure your pickle is to sign it, like:
>
> cookie = dumps(object)
> secret = 'really secret!'
> hasher = md5.new()
> hasher.update(s ecret)
> hasher.update(c ookie)
> cookie_signatur e = md5.digest()[/color]

That method is vulnerable to an "appending" attack against md5. I'll
spare the gory details, but you should call md5 through the HMAC
module to make the signature instead of using md5 directly. HMAC is
designed to stop that attack.
[color=blue]
> You may then wish to base64 encode both (.encode('base6 4')), pop them
> into one value, and you're off. Though I suppose at that point you may
> be hitting the maximum value of a cookie. Hidden fields will work
> nicely, though.[/color]

You could split the session info into several cookies, but in that
situation you should authenticate the whole cookie set with a single
signature. Otherwise someone could paste together several cookies
from separate sessions, and possibly confuse your server.

Re: Securing 'pickle'

[color=blue]
> On Thu, 2003-07-10 at 20:20, David McNab wrote:[color=green]
> > I'm writing a web app framework which stores pickles in client cookies.
> >
> > The obvious security risk is that some 5cr1p7 X1ddi35 will inevitably try
> > tampering with the cookie and malforming it in an attempt to get the
> > server-side python code to run arbitrary code, or something similarly
> > undesirable.
> >
> > To protect against this, I've subclassed pickle.Unpickle r, and added
> > overrides of the methods load_global, load_inst, load_obj and find_class.[/color]
>
> A much easier way to secure your pickle is to sign it, like:
>
> cookie = dumps(object)
> secret = 'really secret!'
> hasher = md5.new()
> hasher.update(s ecret)
> hasher.update(c ookie)
> cookie_signatur e = md5.digest()
>
> You may then wish to base64 encode both (.encode('base6 4')), pop
> them into one value, and you're off. Though I suppose at that point
> you may be hitting the maximum value of a cookie. Hidden fields
> will work nicely, though.
>
> Decoding and verifying is an exercise left to the reader.[/color]

That is exactly what Albatross does with pickles sent to the browser.
In case it is interesting to anyone, here is the class that does the
work of signing and checking the sign.

- Dave

class PickleSignMixin :

def __init__(self, secret):
self.__secret = secret

def pickle_sign(sel f, text):
m = md5.new()
m.update(self._ _secret)
m.update(text)
text = m.digest() + text
return text

def pickle_unsign(s elf, text):
digest = text[:16]
text = text[16:]
m = md5.new()
m.update(self._ _secret)
m.update(text)
if m.digest() == digest:
return text
return ''


--

Re: Securing 'pickle'

Dave Cole <djc@object-craft.com.au> writes:[color=blue]
> I have been googling for information on the "appending" attack against
> md5 and cannot find anything that clearly describes it. Do you have
> any links handy?[/color]

I think RFC 2104 (the HMAC spec) might describe it. Basically, think
about how md5 works. You load the md5 context with the secret key
(say 20 bytes) then your data (say 20 bytes), then some padding to
fill the 64 byte context, and run the compression function:

md5_compress(ke y + data + 24 bytes of padding)

Call the 24 padding bytes P. They are just 16 0's plus an 8 byte
length, iirc.

The hash output is just the md5 chaining variables after running the
compression function.

Now look at the 100 byte string

E = your data + P (same as above) + 36 bytes of evil stuff

Even without knowing your secret key, if the attacker knows your data
(which may not be secret), and md5(key+data) (which you've included in
the cookie), he can compute the signature of E. It's just the result
of running the compression function on his evil stuff plus appropriate
additional padding, with the chaining variables set to the original
md5 hash that you already sent him.

This is not really a failure of md5, which is supposed to be a message
digest algorithm, not a MAC. Rather, the authentication fails because
md5 is being used in a way it was not intended to be used.

The solution is to use HMAC. See RFC 2104 for details.

Re: Securing 'pickle'

Ian Bicking <ianb@colorstud y.com> writes:
[...][color=blue]
> Security isn't a big deal -- or rather, securing cookies isn't a big
> deal.[/color]

I don't understand. The problem is that pickles can be constructed
that can damage systems when unpickled, is that right? If that's
true, then surely unpickling cookie data is unsafe, because stuff
coming in from the network has to be regarded as malevolent. Are you
saying that web server environments are sufficiently-well bolted down
that no pickle attack will work? But belt-and-braces is the best
policy, isn't it?

[color=blue]
> and IE has a
> bug where you can't redirect and set a cookie at the same time, which
> can really drive you crazy if you don't know about it.[/color]
[...]

Hah. There's a slight irony there, given that they fought against
restrictions on setting cookies from 'unverified' third parties when
the (more-or-less stillborn) cookie RFCs were being written. So they
argue against that, then end up partially implementing it by
accident...


John

Re: Securing 'pickle'

jjl@pobox.com (John J. Lee) writes:[color=blue][color=green]
> > Security isn't a big deal -- or rather, securing cookies isn't a big
> > deal.[/color]
>
> I don't understand. The problem is that pickles can be constructed
> that can damage systems when unpickled, is that right? If that's
> true, then surely unpickling cookie data is unsafe, because stuff
> coming in from the network has to be regarded as malevolent. Are you
> saying that web server environments are sufficiently-well bolted down
> that no pickle attack will work? But belt-and-braces is the best
> policy, isn't it?[/color]

The point is that you can use cryptographic signatures to make sure
any cookie you receive is one that the server actually sent, before
deciding to unpickle it. That means if the attacker constructs a
malicious cookie, you never unpickle it.

Re: Securing 'pickle'

Paul Rubin wrote:
[color=blue]
> My suggestion is to
> authenticate the cookies with a cryptographic checksum and verify the
> authentication before deserializing the cookies. That's probably the
> simplest approach. Keeping session info on a multi-process server (or
> worse, a multi-server network) needs some kind of concurrent storage
> mechanism.[/color]

Paul,

Do you mean transmit the checksum to the client with the cookie? And
check that they match when the cookie and checksum come back?

Or is the checksum stored on the server, in some form of lookup
dictionary keyed by some user session identifier?

regards,

--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan

Re: Securing 'pickle'

Alan Kennedy <alanmk@hotmail .com> writes:[color=blue]
> Do you mean transmit the checksum to the client with the cookie? And
> check that they match when the cookie and checksum come back?[/color]

Yes. See other posts in the thread for sample code.
[color=blue]
> Or is the checksum stored on the server, in some form of lookup
> dictionary keyed by some user session identifier?[/color]

If you have a convenient way to do that, it's best to just send a
session number in the cookie, and keep all the session data on the
server. Then you don't ever have to unpickle anything.

Re: Securing 'pickle'

>[color=blue]
>[color=green]
>>Security isn't a big deal -- or rather, securing cookies isn't a big
>>deal.
>>
>>[/color]
>
>I don't understand. The problem is that pickles can be constructed
>that can damage systems when unpickled, is that right? If that's
>true, then surely unpickling cookie data is unsafe, because stuff
>coming in from the network has to be regarded as malevolent. Are you
>saying that web server environments are sufficiently-well bolted down
>that no pickle attack will work? But belt-and-braces is the best
>policy, isn't it?
>[/color]

I'm sorry, I just caught this thread and I don't know your problem very
well.
I'm extensively using this (see below). It is unable to pickle class
instances but
you won't accidentally run constructors. I think it is safe for
sending/receiving data.
I did not try to publish this yet but it would be great to know if this
is safe or not
so please make comments.

Laci 1.0

import cStringIO
import cPickle

def dumps(obj):
f = cStringIO.Strin gIO()
p = cPickle.Pickler (f,1)
p.dump(obj)
return f.getvalue()

def loads(s):
f = cStringIO.Strin gIO(s)
p = cPickle.Unpickl er(f)
p.find_global = None
return p.load()

Re: Securing 'pickle'

>[color=blue]
>
>
>[color=green]
>>My suggestion is to
>>authenticat e the cookies with a cryptographic checksum and verify the
>>authenticatio n before deserializing the cookies. That's probably the
>>simplest approach. Keeping session info on a multi-process server (or
>>worse, a multi-server network) needs some kind of concurrent storage
>>mechanism.
>>
>>[/color]
>
>Paul,
>
>Do you mean transmit the checksum to the client with the cookie? And
>check that they match when the cookie and checksum come back?
>
>Or is the checksum stored on the server, in some form of lookup
>dictionary keyed by some user session identifier?
>[/color]
I think he wanted to write a digital signature instead. Right?

Laci 1.0