Watching a website for periodic outages

I'm needing to monitor a web application for periodic outages and log the
duration.

I originally tried this with SafariWatir as the login is javascript so I
can't use Mechanize. However Watir seems to give up if the page takes
longer than 10 seconds to load, normally I would just catch this error and
retry until successful. However after running for a little while the
browser seems to hang up, I can reach the app from another browser. Plus
the documentation for SafariWatir seems to be non-existant.

Can anyone think of a better way to do this? Or point me to some
SafariWatir documentation? What ever I use needs to be able to handle
Javascript and cookies.

···

--
"Hey brother Christian with your high and mighty errand, Your actions speak
so loud, I can’t hear a word you’re saying."

-Greg Graffin (Bad Religion)

I'm needing to monitor a web application for periodic outages and log the
duration.

I originally tried this with SafariWatir as the login is javascript so I
can't use Mechanize. However Watir seems to give up if the page takes
longer than 10 seconds to load, normally I would just catch this error

and

retry until successful. However after running for a little while the
browser seems to hang up, I can reach the app from another browser. Plus
the documentation for SafariWatir seems to be non-existant.

Can anyone think of a better way to do this? Or point me to some
SafariWatir documentation? What ever I use needs to be able to handle
Javascript and cookies.

Sounds like you're looking for a monitoring solution like Nagios or Zenoss,
which would keep historic data of the status of the site, and produce
alerts when it's unavailable.

- Kyle

···

On Fri, 24 Jul 2009 02:52:37 +0900, Glen Holcomb <damnbigman@gmail.com> wrote:

--
- Kyle

Other than using a monitoring package like Zenoss or Natgios like Kyle said?
Sure, you could use the ruby HTTP classes.
http://ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html

Read the examples, they will prettymuch show you exactly what you need
to do to get a page. The read more to learn how to set your timeout,
etc.

Put it in a loop, catch the exceptions, toss it in a log file with
stuff like log4r, or just plain old file output.

Enjoy!
--Kyle

···

On Thu, Jul 23, 2009 at 12:52 PM, Glen Holcomb<damnbigman@gmail.com> wrote:

I'm needing to monitor a web application for periodic outages and log the
duration.

I originally tried this with SafariWatir as the login is javascript so I
can't use Mechanize. However Watir seems to give up if the page takes

OK, re-read. _Why_ does it need to handle javascript and cookies?

To monitor if it's up and responding, you really only need to get
straight http, check that you didn't get an error code (401 404 etc),
and parse the resulting html to make sure it's not an error message
from your application.

Now, if you really need something that will handle cookies, and do
logins etc (but not javascript), you_could_ use JMeter, but it's
probably overkill to use it just to see if a site is up...

···

On Thu, Jul 23, 2009 at 12:52 PM, Glen Holcomb<damnbigman@gmail.com> wrote:

Can anyone think of a better way to do this? Or point me to some
SafariWatir documentation? What ever I use needs to be able to handle
Javascript and cookies.

Damnit, this is going to get confusing.

--Kyle

···

On Thu, Jul 23, 2009 at 1:34 PM, Kyle Smith<lists@askreet.com> wrote:

Sounds like you're looking for a monitoring solution like Nagios or Zenoss,
which would keep historic data of the status of the site, and produce
alerts when it's unavailable.

- Kyle

--
- Kyle

The problem is this:

We have an application that is misbehaving, and despite paying for support
from the vendor we don't actually recieve "support" from the vendor. The
application is serving static content all the time without issue. However
at certain (random) times it will hang when you request anything that isn't
static content. The login page is the only page that is entirely static.

So unfortunately you need to be attempting to log in or already logged in
and requesting "internal" content to notice the issue. Basically I'm
wanting to log when these are occuring and how long they last so we can be
more selective when looking for the cause.

I tried Watir but after a while of clicking on links inside the app Safari
hangs. I was looking at Selenium but the gem didn't seem to install all the
required files.

I need javascript because the vendor decided that the login form should be
javascript so that that one frame can be SSL encrypted while leaving the
rest of the content clear, to increase performance under high load I guess.

I was looking for a Ruby way to get into the app and once there make sure
responses are timely, if not I wanted to log the interval of discontent.

···

On Thu, Jul 23, 2009 at 12:48 PM, Kyle Schmitt <kyleaschmitt@gmail.com>wrote:

On Thu, Jul 23, 2009 at 12:52 PM, Glen Holcomb<damnbigman@gmail.com> > wrote:
> Can anyone think of a better way to do this? Or point me to some
> SafariWatir documentation? What ever I use needs to be able to handle
> Javascript and cookies.

OK, re-read. _Why_ does it need to handle javascript and cookies?

To monitor if it's up and responding, you really only need to get
straight http, check that you didn't get an error code (401 404 etc),
and parse the resulting html to make sure it's not an error message
from your application.

Now, if you really need something that will handle cookies, and do
logins etc (but not javascript), you_could_ use JMeter, but it's
probably overkill to use it just to see if a site is up...

--
"Hey brother Christian with your high and mighty errand, Your actions speak
so loud, I can’t hear a word you’re saying."

-Greg Graffin (Bad Religion)

Try recording and tweaking a JMeter script. It may just work (after a
fashion) without the javascript bits, as long as it's on when you do
the initial recording.

Humm... That becomes difficult.

You could try using Celerity, which is a JRuby equivalent for Watir. In most
cases it
should be API compatible with Watir, but it is based on HttpUnit (a pure
Java HTTP
client for testing) instead of automating a browser.

Its faster than Watir, and likely to be much less memory intensive. It also
has a built
in JavaScript interpreter (Rhino), and has some special features for working
with
SSL enabled sites (its easy to set it up not to verify - under Watir this
behaviour
is not within API control).

As its JRuby, you might get better luck having these processes work for
extended
periods, which might fit your use case better.

regards,
Richard

···

On Thu, Jul 23, 2009 at 8:02 PM, Glen Holcomb <damnbigman@gmail.com> wrote:

I tried Watir but after a while of clicking on links inside the app Safari
hangs. I was looking at Selenium but the gem didn't seem to install all
the
required files.

I need javascript because the vendor decided that the login form should be
javascript so that that one frame can be SSL encrypted while leaving the
rest of the content clear, to increase performance under high load I guess.

Yeah, I was kind of hoping it would be a lot easier than it seems to be.
Watir doesn't seem to be the answer at all. I just noticed a "pause" and
started my Watir script up. It logs in but after about 10 seconds it tries
to click a link despite my explicit instructions not to until content has
loaded. At least I thought they were explicit instructions.

def check_page
  loaded = false

  while !loaded
    if (@last_link == "Employees" and @browser.link(:text, "Tax Forms")) or
(@last_link == "Welcome" and @browser.link(:text, "Alternative
Transportation"))
      check_load_time
      loaded = true
    end
    sleep 15
  end
end

It seems to get out of this loop somehow without content loading. That or
Watir gets content but the browser never renders the content.

···

On Thu, Jul 23, 2009 at 1:41 PM, Kyle Schmitt <kyleaschmitt@gmail.com>wrote:

Try recording and tweaking a JMeter script. It may just work (after a
fashion) without the javascript bits, as long as it's on when you do
the initial recording.

Humm... That becomes difficult.

--
"Hey brother Christian with your high and mighty errand, Your actions speak
so loud, I can’t hear a word you’re saying."

-Greg Graffin (Bad Religion)

Try recording and tweaking a JMeter script. It may just work (after a
fashion) without the javascript bits, as long as it's on when you do
the initial recording.

Humm... That becomes difficult.

Yeah, I was kind of hoping it would be a lot easier than it seems to be.
Watir doesn't seem to be the answer at all. I just noticed a "pause" and
started my Watir script up. It logs in but after about 10 seconds it tries
to click a link despite my explicit instructions not to until content has
loaded. At least I thought they were explicit instructions.

def check_page
loaded = false

while !loaded
if (@last_link == "Employees" and @browser.link(:text, "Tax Forms")) or
(@last_link == "Welcome" and @browser.link(:text, "Alternative
Transportation"))
check_load_time
loaded = true
end
sleep 15
end
end

It seems to get out of this loop somehow without content loading. That or
Watir gets content but the browser never renders the content.

--
"Hey brother Christian with your high and mighty errand, Your actions speak
so loud, I can’t hear a word you’re saying."

-Greg Graffin (Bad Religion)

···

On Thu, Jul 23, 2009 at 10:50 PM, Glen Holcomb<damnbigman@gmail.com> wrote:

On Thu, Jul 23, 2009 at 1:41 PM, Kyle Schmitt <kyleaschmitt@gmail.com>wrote:

They're not explicit instructions. You're merely testing for the value
of @last_link. If @last_link is available, you're executing
@browser.link which follows the link. If you want explicit, use
@browser.wait().

-- Mark.

···

On Jul 23, 4:50 pm, Glen Holcomb <damnbig...@gmail.com> wrote:

I just noticed a "pause" and
started my Watir script up. It logs in but after about 10 seconds it tries
to click a link despite my explicit instructions not to until content has
loaded. At least I thought they were explicit instructions.

Sorry guys, I accidentally pushed reply or something.

Hmmm...

@browser.wait() gives me the following error:

undefined method `wait' for #<Watir::Safari:0x1213184> (NoMethodError)

It would appear I can't do that with SafariWatir

···

On Sat, Jul 25, 2009 at 6:50 PM, Mark Thomas <mark@thomaszone.com> wrote:

On Jul 23, 4:50 pm, Glen Holcomb <damnbig...@gmail.com> wrote:
> I just noticed a "pause" and
> started my Watir script up. It logs in but after about 10 seconds it
tries
> to click a link despite my explicit instructions not to until content has
> loaded. At least I thought they were explicit instructions.

They're not explicit instructions. You're merely testing for the value
of @last_link. If @last_link is available, you're executing
@browser.link which follows the link. If you want explicit, use
@browser.wait().

-- Mark.

--
"Hey brother Christian with your high and mighty errand, Your actions speak
so loud, I can’t hear a word you’re saying."

-Greg Graffin (Bad Religion)

Also, I was under the impression that @browser.link(:text, "link_name")
would only return true if that could be found on the page. Is that the
wrong thing to expect?

···

On Sat, Jul 25, 2009 at 6:50 PM, Mark Thomas <mark@thomaszone.com> wrote:

On Jul 23, 4:50 pm, Glen Holcomb <damnbig...@gmail.com> wrote:
> I just noticed a "pause" and
> started my Watir script up. It logs in but after about 10 seconds it
tries
> to click a link despite my explicit instructions not to until content has
> loaded. At least I thought they were explicit instructions.

They're not explicit instructions. You're merely testing for the value
of @last_link. If @last_link is available, you're executing
@browser.link which follows the link. If you want explicit, use
@browser.wait().

-- Mark.

--
"Hey brother Christian with your high and mighty errand, Your actions speak
so loud, I can’t hear a word you’re saying."

-Greg Graffin (Bad Religion)