# Proxy Server **URL:** \ **Category:** ruby-talk\ **Created:** [11 September 2002 17:36 UTC](https://rubytalk.org/t/proxy-server/1702 "2002-09-11T17:36:37Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Alan\_Chen2](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@Alan\_Chen2](https://rubytalk.org/u/Alan_Chen2)\ **Post date:** [11 September 2002 17:36 UTC](https://rubytalk.org/t/proxy-server/1702/1 "2002-09-11T17:36:37Z")
I’ve been playing around with a project that’s a proxy and cache server for a Debian apt repository. (apt-proxy in ruby basically) I’ve got a simple example working using Webrick, but have one annoying problem I’d like to solve. When an apt client requests a package file from the proxy, the client receives no feedback until the proxy has completely downloaded the file from the backend repository. I’d like to start sending the file to the client while the backend download is ongoing. I’m hoping this would also have the side benefit of not having to load the entire package file into interpreter memory. The amazing thing is the the original Debian apt-proxy code does this in a shell-script. I could adopt the apt-proxy methods and use system calls to perform this, but I was hoping for a more rubyesque solution. Any ideas? - alan > **···** > > – > Alan Chen > Digikata LLC > > > **[Digikata](http://www.digikata.com)** > > > > An engineer understanding the world... ---
**Author:** ![Tom\_Sawyer](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@Tom\_Sawyer](https://rubytalk.org/u/Tom_Sawyer)\ **Post date:** [11 September 2002 17:51 UTC](https://rubytalk.org/t/proxy-server/1702/2 "2002-09-11T17:51:02Z")
quick guess: read file only in chunks, pass off each chuck to another thread (don’t ask me much about this, i’m not sure. check out mutexes) this other thread of course then passes it on to the user. like i said my first thoughts. cheers transami > **···** > > On Wed, 2002-09-11 at 11:36, Alan Chen wrote: > > > I’ve been playing around with a project that’s a proxy and cache > > server for a Debian apt repository. (apt-proxy in ruby basically) I’ve > > got a simple example working using Webrick, but have one annoying > > problem I’d like to solve. > > > > When an apt client requests a package file from the proxy, the client > > receives no feedback until the proxy has completely downloaded the > > file from the backend repository. I’d like to start sending the file > > to the client while the backend download is ongoing. I’m hoping this > > would also have the side benefit of not having to load the entire > > package file into interpreter memory. > > > > The amazing thing is the the original Debian apt-proxy code does this > > in a shell-script. I could adopt the apt-proxy methods and use > > system calls to perform this, but I was hoping for a more rubyesque > > solution. > > > > Any ideas? > > > > - alan > > > > – > > Alan Chen > > Digikata LLC > > [http://digikata.com](http://digikata.com) > > – > tom sawyer, aka transami > [transami@transami.net](mailto:transami@transami.net) ---
**Author:** ![GOTO\_Kentaro1](https://avatars.discourse-cdn.com/v4/letter/g/e5b9ba/32.png) [@GOTO\_Kentaro1](https://rubytalk.org/u/GOTO_Kentaro1)\ **Post date:** [12 September 2002 02:20 UTC](https://rubytalk.org/t/proxy-server/1702/3 "2002-09-12T02:20:46Z")
A difficulty come from Net::HTTP#get, which is used by WEBrick::HTTPProxy internally. HTTP#get{} doesn’t tell http response header to its given block. So WEBrick can’t return the header to user agent until HTTP#get returns. I’m planing to modify WEBrick#HTTPProxy to use WEBrick’s own HTTPClient instead of Net::HTTP to solve this problem. But anything about its detail have not decided still now. If you have any idea on design of HTTPClient, please discuss at webricken list. [http://www.notwork.org/ipr/webrick/#ml](http://www.notwork.org/ipr/webrick/#ml) Thanks, – Gotoken > **···** > > At Thu, 12 Sep 2002 02:36:37 +0900, Alan Chen wrote: > > > When an apt client requests a package file from the proxy, the client > > receives no feedback until the proxy has completely downloaded the > > file from the backend repository. I’d like to start sending the file > > to the client while the backend download is ongoing. I’m hoping this > > would also have the side benefit of not having to load the entire > > package file into interpreter memory. ---
**Author:** ![Alan\_Chen2](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@Alan\_Chen2](https://rubytalk.org/u/Alan_Chen2)\ **Post date:** [11 September 2002 18:21 UTC](https://rubytalk.org/t/proxy-server/1702/4 "2002-09-11T18:21:02Z")
Sorry if I was unclear, you’ve caught onto the general idea, but what I was mostly looking for was a ruby class or library which already handles this. > **···** > > On Thu, Sep 12, 2002 at 02:51:02AM +0900, Tom Sawyer wrote: > > > quick guess: > > > > read file only in chunks, pass off each chuck to another thread (don’t > > ask me much about this, i’m not sure. check out mutexes) this other > > thread of course then passes it on to the user. > > > > like i said my first thoughts. > > > > cheers > > transami > > – > Alan Chen > Digikata LLC > [http://digikata.com](http://digikata.com) ---
**Author:** ![Alan\_Chen2](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@Alan\_Chen2](https://rubytalk.org/u/Alan_Chen2)\ **Post date:** [12 September 2002 17:09 UTC](https://rubytalk.org/t/proxy-server/1702/5 "2002-09-12T17:09:36Z")
I’ve been peeking at libcurl which implements an optional callback every time a block of data is received on the socket. This allows you to write your own callback to route the data block. So we could initiate a download, and have a callback write the received blocks to multiple destinations at once - in my case, to the file system and the requesting apt-get client. > **···** > > On Thu, Sep 12, 2002 at 11:20:46AM +0900, GOTO Kentaro wrote: > > > I’m planing to modify WEBrick#HTTPProxy to use WEBrick’s own > > HTTPClient instead of Net::HTTP to solve this problem. But anything > > about its detail have not decided still now. If you have any idea on > > design of HTTPClient, please discuss at webricken list. > > [http://www.notwork.org/ipr/webrick/#ml](http://www.notwork.org/ipr/webrick/#ml) > > > > Thanks, > > > > – Gotoken > > – > Alan Chen > Digikata LLC > [http://digikata.com](http://digikata.com)