SNF Labs

i was bored before i even began

The sequel to Shoplifters, Unite will have to wait. I began rewriting it yesterday and had completed the initial basic work — creating song and request classes, displaying a rudimentary index page, etc. — when I moved on to implementing downloads and quickly hit a wall. The download code in Shoplifters works like this:

  1. verify that user is allowed to download
  2. check that requested file is available
  3. output HTTP headers
  4. output the file using a call to readfile()

It turns out that the last step no longer works. In PHP 5.0.4 and only PHP 5.0.4, readfile() has a bug that causes it to halt output at about 1.9 MB. This means that only the tiniest of files can be downloaded successfully from Shoplifters.

I sent a note to Dreamhost support and even created a demonstration page for the bug so that they can see what I’m talking about. All they have to do to fix the issue is to upgrade PHP to any subsequent version, but they’re very conservative about PHP upgrades, so there’s no telling when that might happen.

There are things I could do to work around the problem, but I don’t like any of the options that I’ve been able to come up with:

  1. link directly to download files instead of passing them through an intermediary script
  2. downgrade to PHP 4
  3. write Shoplifters 2 in some other language

Posted December 01, 2005 by cobra libre | shoplifters

 


Comments

posted 2005-12-02 02:49:32 by fhazel:

that’s kind of a crippling blow, isn’t it. crap.

posted 2005-12-05 18:09:33 by cobra libre:

They wrote back and suggested that I compile my own copy of PHP, because it could be months before they upgrade to a newer version.

posted 2005-12-05 21:40:22 by fhazel:

i don’t suppose doing that would let you increase the uploaded file size limit to something bigger than 6-8 megs would it?

posted 2005-12-05 21:52:05 by cobra libre:

Holy shit. That’s a good question.

posted 2005-12-05 22:12:33 by fhazel:

it’s controlled by a php.ini setting, isn’t it? or is there also something in apache that affects it.

posted 2005-12-06 13:56:42 by cobra libre:

I should have said “good call” instead of “good question.” It’s definitely controlled by a php.ini setting. We still want to keep the file size limit down to something reasonable because I think that Dreamhost uses a process reaper that will kill anything that’s been running for too long. Anyway, we can experiment to see what works.

Without going into the details, compiling PHP has been a real pain in the ass so far, unfortunately, but it’s compiling again as I speak, and I think it’s going to work.

The next question is whether or not FastCGI will work. If it doesn’t, then the performance is going to be poor, and I’ll probably only use the custom PHP binary for uploading and downloading.

posted 2005-12-06 15:52:57 by cobra libre:

Okay, after several iterations, I was finally able to compile a working PHP 5.1.1. A copy of the binary resides in ~/www/shoplifters/bin.

Downloads work perfectly now. When I have some spare time one of these evenings, I’ll work on the upload code.

The performance seems very satisfactory, so that’s not a concern, I guess. I had to disable some features that Dreamhost normally enables to get the thing to compile, so I’d continue to use the official Dreamhost PHP unless you have a specific reason not to. I’d also prefer to abandon the custom compiled PHP as soon as Dreamhost updates their install; otherwise, I’ll have to continually worry about keeping this PHP build (and its constituent libraries) updated for security issues.

posted 2005-12-07 00:27:19 by fhazel:

hahaha, i shouldn’t have posted a statement in the form of a question. anyway, that’s cool! what did you set the upload limit to? even upping it to 9 or 10 megs would make a big difference.


Post a comment

You are not currently logged in. You need to have a registered SNF account to post a comment. (log in)