Semi-OT: Web issues

There’s a disturbing absence of Ruby code
in this post, but it’s somewhat on-topic
as I intend to write a Ruby app for this.

I’ve been playing with CafePress.com lately.
This is an “on-demand” printing place that
lets people upload their images and put
them on mugs, frisbees, clocks, shirts, and
so on.

If you want to look at my problem in any
detail, you’ll have to spend five minutes
signing up for a free store. The relevant
portions of the site require a login.

And no, I don’t blame you if you don’t
bother.

The process is all web-based and a little
clunky. I’ve thought about writing a tool
to assist.

They have a page where all your graphics
are stored, called the "image basket."
To add a graphic, you browse to it on
your hard drive and it’s uploaded.

Difficulties:

  1. First of all, it won’t work if your
    login times out. I’m not even certain
    how it knows you’re logged in. With any
    luck, it’s some kind of timestamped
    token on the server end. If it’s a cookie
    on my end, that would complicate my code.

  2. I started by taking the image-upload
    HTML and stripping it to its bare minimum.
    However: This “bare minimum” doesn’t work.
    (Even when I’ve just logged in via the
    browser.) I tried using Javascript to
    fiddle with the referrer, thinking it
    might be checking that. No luck so far.
    HTML shown below.

Of course, what I’d really like is just
a little Ruby method that will take a
local filename and upload it programmatically.
(I hate that word. But you know what I mean.)

Cheers,
Hal

I agree to the
statements made above.

Image File

aspx is ASP.NET – I’m a little familiar with it. It’s possible you stripped
out some of the html that communicates your session information back to the
server. ASP.NET pages can be setup to automagically handle a lot of session
state for you, it does this by embedding some session identifying stuff in
the html.

Chris

Difficulties:

  1. First of all, it won’t work if your
    login times out. I’m not even certain
    how it knows you’re logged in. With any
    luck, it’s some kind of timestamped
    token on the server end. If it’s a cookie
    on my end, that would complicate my code.

  2. I started by taking the image-upload
    HTML and stripping it to its bare minimum.
    However: This “bare minimum” doesn’t work.
    (Even when I’ve just logged in via the
    browser.) I tried using Javascript to
    fiddle with the referrer, thinking it
    might be checking that. No luck so far.
    HTML shown below.

Of course, what I’d really like is just
a little Ruby method that will take a
local filename and upload it programmatically.
(I hate that word. But you know what I mean.)

Cheers,
Hal

Some thoughts:

  1. The aspx pages are using session cookies. I tried uploading an image from
    an IE-based browser that goes through a proxy filter, and even though I
    thought I had everything turned off all filtering, it kept failing. I tried
    again using Mozilla, with no proxy stuff, and it worked fine. I set Mozilla
    to prompt me for cookies, but never saw anything. Odd. But when I go to
    “Managed stored cookies”, I find this a cookie named “ASP.NET_SessionId” for
    www.cafepress.com

  2. The site checks the HTTP_REFERER header. Harder to test. You’d have to
    spoof this in your code.

I think you need to add cookie handling to your script. I’d be worried if
anybody could just upload files to a URL without some sort of ID process in
place. Besides, the HTML you showed doesn’t contain any information about
whose pictures are being uploaded, or where they should go, so I think this
is stored in cookies.

James

If this isn't the most pathetic guilt trip I'd ever heard! :slight_smile:

Since I'd just spent the last two days writing web screen-scrapers
as a favor to a friend -- part of which was defeating some website's
attempt to limit access to some portions of their site -- your challenge
wasn't very tough. :slight_smile:

Here's the skinny:

The CafePress account side can either be accessed via SSL or non-SSL.

Here are the cookies that the site sets:

Cookie: fcP=C=0&T=1032397984660; ASP.NET_SessionId=ptqg1xrfd5ivxinux3ai2art

So, we've got "fcP" which is set to "C=0" and "T" which is set to what
looks like a Unix timestamp:

$ date
Wed Sep 18 21:22:48 EDT 2002

$ ruby -e 'p Time.at(1032397984660)'
-e:1:in `at': bignum too big to convert into `int' (RangeError)
  from -e:1

Hmm. It definitely looks like Unix time, but too many digits.
Lets take the least significant ones off:

$ ruby -e 'p Time.at(1032397984)'
Wed Sep 18 21:13:04 EDT 2002

There you go.

So, there's some server-side state that's being maintained using the
ASP.NET_SessionId cookie to associate your browser to the server-side
session. Then there's some client-side timestamp and some other misc.
cookie.

So, you'll have to write your auto-uploader to hit the CafePress login
page in order to get the ASP.NET_SessionId. Once you've got that, then
go on your merry way.

Ruby's Net::HTTP makes this pretty easy.

-- Dossy

···

On 2002.09.19, Hal E. Fulton <hal9000@hypermetrics.com> wrote:

And no, I don't blame you if you don't bother.

--
Dossy Shiobara mail: dossy@panoptic.com
Panoptic Computer Network web: http://www.panoptic.com/
  "He realized the fastest way to change is to laugh at your own
    folly -- then you can let go and quickly move on." (p. 70)

aspx is ASP.NET – I’m a little familiar with it. It’s possible you
stripped
out some of the html that communicates your session information back to
the
server. ASP.NET pages can be setup to automagically handle a lot of
session
state for you, it does this by embedding some session identifying stuff in
the html.

Well, I didn’t see much out of the ordinary. There was an
external “commonscripts.js” that I haven’t looked at yet.
There’s an external style sheet that I hope has nothing
to do with aspx.

And there’s this cryptic bit of Javascript at the
bottom, but I guessed it might be related to tracking or
something. Hmm.

Hal

···

----- Original Message -----
From: “Chris Morris” chrismo@clabs.org
To: “ruby-talk ML” ruby-talk@ruby-lang.org
Sent: Wednesday, September 18, 2002 5:41 PM
Subject: Re: Semi-OT: Web issues

  1. The site checks the HTTP_REFERER header. Harder to test.
    You’d have to spoof this in your code.

If it were found to be the case. I was just guessing here. Cookies look
like a likely culprit.

James

(snippage)

I think you need to add cookie handling to your script. I’d be worried if
anybody could just upload files to a URL without some sort of ID process
in
place. Besides, the HTML you showed doesn’t contain any information about
whose pictures are being uploaded, or where they should go, so I think
this
is stored in cookies.

I should have realized that sooner. They have to identify
me somehow. Probably not by IP. Though I guess that’s
theoretically possible.

Thanks for looking at this. Dossy seems to have
shown that it’s all in the cookies, and apparently
there’s no referrer interaction.

thanx*10**6,

Hal

···

----- Original Message -----
From: " JamesBritt" james@jamesbritt.com
To: “ruby-talk ML” ruby-talk@ruby-lang.org
Sent: Wednesday, September 18, 2002 8:23 PM
Subject: RE: Semi-OT: Web issues

And no, I don’t blame you if you don’t bother.

If this isn’t the most pathetic guilt trip I’d ever heard! :slight_smile:

:slight_smile: But it apparently worked.

(much interesting snippage)

Thanks, Dossy! I replied offline.

Trying to remember where you’re located. Are you by
any chance going to the RubyConf?

Hal

···

----- Original Message -----
From: “Dossy” dossy@panoptic.com
To: “ruby-talk ML” ruby-talk@ruby-lang.org; “Hal E. Fulton”
hal9000@hypermetrics.com
Cc: “ruby-talk ML” ruby-talk@ruby-lang.org
Sent: Wednesday, September 18, 2002 8:26 PM
Subject: Re: Semi-OT: Web issues

On 2002.09.19, Hal E. Fulton hal9000@hypermetrics.com wrote:

And there’s this cryptic bit of Javascript at the
bottom, but I guessed it might be related to tracking or
something. Hmm.

Please delete that code! :slight_smile:

It’s code to dynamically alter the HTML DOM and insert a script ref so that
an evil pop-up window can haunt users.

James

···

Hal

No, unfortunately.

I’m in New Jersey. If David can arrange the next RubyConf to be
held in New Jersey, there might be a chance I could go. :wink:

– Dossy

···

On 2002.09.19, Hal E. Fulton hal9000@hypermetrics.com wrote:

Trying to remember where you’re located. Are you by
any chance going to the RubyConf?


Dossy Shiobara mail: dossy@panoptic.com
Panoptic Computer Network web: http://www.panoptic.com/
“He realized the fastest way to change is to laugh at your own
folly – then you can let go and quickly move on.” (p. 70)

And there’s this cryptic bit of Javascript at the
bottom, but I guessed it might be related to tracking or
something. Hmm.

Please delete that code! :slight_smile:

It’s code to dynamically alter the HTML DOM and insert a script ref so
that
an evil pop-up window can haunt users.

Oh, yes, I see. Figured it was marketing, but it’s
advertising. :slight_smile:

At any rate, I don’t see anything that looks
asp-specific in the HTML. Sigh.

Anyone got a clue?

HF

···

----- Original Message -----
From: " JamesBritt" james@jamesbritt.com
To: “ruby-talk ML” ruby-talk@ruby-lang.org
Sent: Wednesday, September 18, 2002 6:53 PM
Subject: RE: Semi-OT: Web issues

The site does set a session cookie, ASP_NET.Session_id or somesuch.
How does your current try fail? Is the server refusing to accept
your file?

···

On Thu, Sep 19, 2002 at 09:31:05AM +0900, Hal E. Fulton wrote:

Anyone got a clue?

HF


Alan Chen
Digikata LLC
http://digikata.com

When I try my stripped HTML in the browser,
it redirects me to:

http://crunch.cafepress.com/errorpage.aspx?aspxerrorpath=/uploadimage.aspx

which says:

We’re Sorry, You’ve Encountered an Error!

You’ve received this message because of the following possibilities:

  1. Slow Internet connectivity speed at this time
  2. Server timed out due to heavy load or traffic
  3. Data has been lost by your browser
  4. Internet interference
    Please try to submit the previous page again. If the problem persists, try
    launching a new browser or reconnecting to the internet.

I haven’t tried writing code to do it yet, as I’m
uncertain of the nuances of coding it.

Hal

···

----- Original Message -----
From: “Alan Chen” alan@digikata.com
To: “ruby-talk ML” ruby-talk@ruby-lang.org
Sent: Wednesday, September 18, 2002 7:43 PM
Subject: Re: Semi-OT: Web issues

The site does set a session cookie, ASP_NET.Session_id or somesuch.
How does your current try fail? Is the server refusing to accept
your file?