Instiki with SQL backend

(Alex Verhovsky) #1

Hi all,

We’ve got a working (at least to some extent) Instiki implementation
backed by an ActiveRecord backend and SQLite database. It’s in a
Subversion branch http://svn.instiki.org/instiki/branches/instiki-ar/,
revision 360.

If you find yourself thinking “heck, so what?”, one possible consequence
is that one day it will drastically reduce rubyonrails.org downtime and
increase its performance. Not to mention that all the people running
public Instiki instances will get the FastCGI deployment option most of
them are crying for. Besides, Instiki is the most popular end-user
application in the Ruby world :slight_smile:

“Working to some extent” refers to the fact that it passes all available
automated tests (unit, functional and Watir), as well as half an hour of
random poking around, while running on WEBRick with SQLite DB, under
Windows XP. It probably will be heinously slow on any wiki larger than
~100 pages, so we are just half-way through the “make it work, then make
it fast” road. Still need to do performance testing, refactor the
database for speed, define indexes, implement some caching etc. In other
words, all the “wizardry” bits. So, I thought maybe I can get some
wizards with Rails production expertise interested to the point of
getting involved at this stage.

Even if you are not a wizard yet, you are most welcome to try it out and
tell me about the problems you encounter.

Best regards,
Alex

(Julian Leviston) #2

What does instiki run on at present "out of the box"?
Julian

···

On 15/08/2005, at 1:41 PM, Alex Verhovsky wrote:

Hi all,

We've got a working (at least to some extent) Instiki implementation backed by an ActiveRecord backend and SQLite database. It's in a Subversion branch http://svn.instiki.org/instiki/branches/instiki-ar/, revision 360.

If you find yourself thinking "heck, so what?", one possible consequence is that one day it will drastically reduce rubyonrails.org downtime and increase its performance. Not to mention that all the people running public Instiki instances will get the FastCGI deployment option most of them are crying for. Besides, <hype>Instiki is the most popular end-user application in the Ruby world</hype> :slight_smile:

"Working to some extent" refers to the fact that it passes all available automated tests (unit, functional and Watir), as well as half an hour of random poking around, while running on WEBRick with SQLite DB, under Windows XP. It probably will be heinously slow on any wiki larger than ~100 pages, so we are just half-way through the "make it work, then make it fast" road. Still need to do performance testing, refactor the database for speed, define indexes, implement some caching etc. In other words, all the "wizardry" bits. So, I thought maybe I can get some wizards with Rails production expertise interested to the point of getting involved at this stage.

Even if you are not a wizard yet, you are most welcome to try it out and tell me about the problems you encounter.

Best regards,
Alex

(John Evans) #3

OK so its simple. I had a couple of .9x installations, I am moving
them to the latest stable version and so I follow some tweaks and
guides here and there and enable file uploads and so crate and empty
web, go to the new_web/import page, select a zip exported by the same
instiki version, hit import and wait. Then I get the file import
success or whatever and now look at the ‘all pages’ and low and
behold lots of orphaned pages. and lots of missing pages.

The reason? any page with a space for example

I am a page

becomes on export

I+am+a+page.textile

which when reimported becomes

I%26am%26a%26page

which is wrong, oh so so so wrong. So my exports are useless, I cant
g back to .9 easily because of upgrading rby and a bunch of other stuff.

WHY WHY WHY WHY, I really need a fix now I have hundreds of pages
which I need to get back online and looking over the code (I am more
a php guy) I get the jist of whats going on but I dont see a way to
fix it easily. Anyone who can just give me a quick fix, even if its
change this line to this I will be eternally grateful…

perhaps a simple url decode of the filename would do?

(now using latest stable trunk version, latest ruby)

john

(Ben Giddings) #4

Cool stuff. It's great to see instiki gradually becoming a standard Rails
app. Now for my pet question: "Will Instiki soon (or is it now) be
something that can be called by a standard CGI/FCGI implementation?"

I ask because I have a web server that is used a lot (tying up port 80),
and an instiki wiki which started small, but now is used a lot, but people
often forget the port for the wiki, and I'd really prefer them both to be
running under Apache on port 80.

Basically, I think it would be great if there were a growth path from a
quick instiki wiki that you can have up and running in a few minutes to
jot notes, to a full-fledged web app that can serve dozens of requests a
minute.

Ben

(Alex) #5

Julian Leviston wrote:

What does instiki run on at present "out of the box"?

WEBRick / Madeleine (the latter is a persistence layer conceptually similar to Prevayler).

(Ben Myles) #6

jot notes, to a full-fledged web app that can serve dozens of requests a
minute.

I'd hope it was more like dozens of requests a second... at least :slight_smile:

Seriously though, I think the main hurdle with Instiki running on
Lighttpd/Apache was the madeleine dependency. Once Instiki's running
stable and fast on a SQL backend I think it should be a pretty
straightforward process.... maybe I'm wrong though?

Ben Myles
railsapphosting.com

(Alex) #7

Ben Myles wrote:

Seriously though, I think the main hurdle with Instiki running on
Lighttpd/Apache was the madeleine dependency. Once Instiki's running
stable and fast on a SQL backend I think it should be a pretty
straightforward process.... maybe I'm wrong though?

You are absolutely right. In fact, one should be able to host the instiki-ar branch on FastCGI even now (not that I tried it, mind you, but there is no known reason why it wouldn't work).

Re concurrency problems in SQLite. Thanks for the advice. I suppose, we should write it somewhere in the docs that SQLite is for WEBRick only (which is not non-conccurrent, anyway); serious people with serious demands will have to choose between Postgres and MySQL.

Re creation of the database. There is a small program called script/create_db which takes your database.yml and creates the schema from db/*.erbsql files in whatever databases are specified. Myself, I usually define :development and :production databases on Postgres, and :test database on SQLite, pysically hosted on a RAMDrive (this speeds up tests by a factor of 3). The script is not integrated with the build yet, sorry.

Alex

(Xavier Noria) #8

Which post talked about concurrency problems with SQLite? Did the subject change?

AFAIK the SQLite C driver takes care of concurrency:

     http://www.sqlite.org/lockingv3.html

-- fxn

···

On Aug 16, 2005, at 6:38, Alex Verhovsky wrote:

Re concurrency problems in SQLite. Thanks for the advice. I suppose, we should write it somewhere in the docs that SQLite is for WEBRick only (which is not non-conccurrent, anyway); serious people with serious demands will have to choose between Postgres and MySQL.

(Alex) #9

Xavier Noria wrote:

Re concurrency problems in SQLite. Thanks for the advice. I suppose, we should write it somewhere in the docs that SQLite is for WEBRick only (which is not non-conccurrent, anyway); serious people with serious demands will have to choose between Postgres and MySQL.

Which post talked about concurrency problems with SQLite? Did the subject change?
AFAIK the SQLite C driver takes care of concurrency:

Oh, I guess, it's been from a thread on Rails ML. Someone said there (probably referring to SQLite 2) that it had concurrency problems.
Having thought about it, there are less than a hundred inserts/updates per day even on fairly active wikis, so concurrency problems may not be much of an issue in practice.

And thanks for the pointer about SQLite3 improved concurrency handling - I'm really starting to like version 3 more and more :slight_smile:

Alex

···

On Aug 16, 2005, at 6:38, Alex Verhovsky wrote: