Join Limit

Regarding the problem where we have a limit of 32 joins, this is the change I recently made to ticket 1516:

************************************************************************************************

 Jeff and I have have been able to reason that any attempt to reduce the
 number of joins must be done in the database side of things; i.e. you
 cannot split the queries, execute the sub-queries then and use set
 theoretic operations to build the resulting set yourself.  [This is because
you need to be able to do orderings and you will need the target table to
be joined, but you don’t know what the target table is so all the joins need
to be present. You could probably do some work to figure this out, but
it’s really hacky and not what good programmers do.]

 We have also reasoned mathematically that, with the current database
 schema, it is not possible to reduce the number of joins by more than a
 factor of two.  So the worst case filter limit would be 16 filters.

 Options:

  * Change the database schema to have fewer tables and joins
  * Locate the join limit and change it

 The second option is clearly retarded.  Moreover, the limit is set by
 SQLite, not Alchemy, so it “can’t” be done.

 We tried changing the schema and discovered that the one we have is quite
 elegant and we run into much greater problems if we use anything
 different.  Besides, changing the schema will most likely only give a
 constant multiplicative benefit; i.e. we can’t do infinitely many joins
 without changing the query mechanism.

************************************************************************************************

In high school I was naive enough to think that programmers didn’t need to know much math. Boy was I wrong. But Jeff says that the problem might not occur in Postgres… but we know there are portals using SQLite.

What to do?

I’ve made this a low priority problem and will move on to test the system starting next week. Luke is also testing, but since I know the ticket system like the back of my hand, I should also test. I will also create a screen cast.

There are some bugs that need to be fixed (e.g. tags, email notification preferences, etc), and some CSS that should be made pretty, but I’m comfortable handing all of this off to anyone else.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: