BIG (up to 45%) performance increase for categorybox.php

July 20, 2003

Seeing as there's not yet too many shows of interest in my proposal, I thought I'd get some code done and do some quick benchmarking.

And I've thrown in a catchy title this time, just for good measure :-)

I have reworked the database parts of the category box to get some these benchmarks done.

I hope these ideas find their way into osC as I think it would help tremendously with performance. If this is not the right channel to get these ideas across to the dev team please let me know where I should post this instead.

I expect there are similar gains to be made with several other parts of the code, there are more places where there are queries inside loops.

Ok, on to the benchmarks, the first results look really promising:

All results in seconds are averages over 10 runs, using the same database and server, doing requests for

http://osc-cvs.denet.nl/catalog/index.php?cPath=1 on an (almost) stock osC install. With default number of categories.

with SHOW_COUNTS 'true':



avg parse time*: 

old code:  .9087

new code:  .6598  (27% faster)



avg query time**:

old code:  .3445

new code:  .1907***    ( 45% faster )

with SHOW_COUNTS 'false':



avg parse time: 

old code:  .6921

new code:  .6159  (11% faster)



avg query time**:

old code:  .2043

new code:  .1755  (14% faster)

The code literally reduces the number of database queries with dozens per page. This is especially true when SHOW_COUNTS is enabled. As you can see the new code is faster (In terms of query time) with SHOW_COUNTS enabled than the old code was without!

Keep in mind that these results where for a default install of osC. (I have been playing around with it, so it's not a complete virgin database, there are some reviews and purchases done)

However if you have more categories in your database the performance gain will most likely be even bigger.

The display code for these results is really sloppy, I just wanted to hack something into osC to get the database timings. So take the overall timing benchmarks with a big grain of salt, I have not tried to implement this code efficiently, or functionally equivalent to the orignal code.

P.S.:

For those of you who read my previous post, I made a mistake there.

The number of queries for the category box are:

SHOW_COUNTS = 'true'

- one query per category that has subcategories.

SHOW_COUNTS = 'false'

- one query per displayed category contaiing subcategories.

Rob

=======

The usual disclaimers apply; These are results on my box not yours, using my data not yours!

* These are of limited value I did not finish the layout code, the code currently constructs a multidimensional array with the category info.

** These are measured by timing each tep_db_query(). So this is excluding any time setting up the connections.

*** Sadly there was one .22 result the others were all in the .18 -.19 range

July 21, 2003

I'm looking for people willing to test/ benchmark my changes on their setup.

What would be extra useful is testing on actual store data.

I've uploaded an archive with instructions to: http://www.cryp2nite.nl/improved_catalog.tar.gz

This is also in the included README, but to avoid confusion:

DO NOT use this on your production site!

It is proof-of-concept code, it will make your site look really, really bad!

Any feedback greatly appreciated.

Rob

July 22, 2003

Seeing as there were some downloads of my code, but no replies yet I uploaded a new version of the benchmark code.

The download is bigger, but testing should be much easier, just unpack adjust the configure.php and you're done.

To everyone that downloaded the previous code and decided it was too much of a hassle, give this a try.

And for your viewing pleasure, a pretty graph of the improvements on an osC demo database.

July 22, 2003

Sorry I forgot:

The old code benchmark code download is still available as:

http://www.cryp2nite.nl/improved_catalog-0.1.tar.gz

new download:

http://www.cryp2nite.nl/improved_catalog.tar.gz

skarra · July 22, 2003

This looks interesting, it would be great to hear what the developers think.

would like to help but definitely out of my depth :lol:

Dave

July 22, 2003

Anyone who has a osC server to play around with can do the benchmarks. I would just like some confirmation of my results.

And some indication of how well this works on setups other than my own.

If it really works as well as I think it will, I'd like to get this on the radar for the dev team as well.

Cheers,

Rob

Daemonj · July 22, 2003

I just downloaded your improved version and will give it a try. I have a few sites that I can test it on. I will post the results later tonight.

July 23, 2003

Hi all,

I noticed a number of you have downloaded the tests, but I don't see any results (yet).

If you have downloaded the code, but are having problems doing the benchmarks please let me know so I can maybe help out.

I would really like some benchmark data other than my own.

Cheers,

Rob

Daemonj · July 23, 2003

Hi Rob :)

I have not had a chance to install and use your code. :(

I am booked up today but will try to give it a shot tonight or tomorrow.

I will let you know the results as soon as I am able to get some time to do it.

July 23, 2003

NP,

Whenever you have the time.

It's just that I registered about dozen downloads and got no response. And because english is not my native language, I figured maybe the included instructions are not as clear as they could be :?

Cheers,

Rob

July 23, 2003

Linux,Konqueror, XS4all,...

Hmm,.. care to help a fellow countryman out?

Sorry, OT, couldn't resist :oops: .

Rob

cwdmark · July 24, 2003

i tried this and it screwed up my navigation... probaby because i custom coded that a while back

it loads fairly fast for me with it true. and i like the fact that it shows the number of products in each category

i guess its personal preference

July 24, 2003

i tried this and it screwed up my navigation... probaby because i custom coded that a while back

I know it messes up the navigation, it's in the readme. I just need some benchmark figures on setups other than my own. If it is significantly faster than the regular menu, I will finish it so you don't see the difference with the old wrt to output. But I would first like to know I'm not wasting my time on this before spending any more time on this.

it loads fairly fast for me with it true. and i like the fact that it shows the number of products in each category

i guess its personal preference

It prints out numbers for 'Parse time' and 'Query time' at the bottom of each page. Could you give me some numbers for the catalog_ori and the catalog_new versions?

What would also be helpful is some figures regarding the number of categories and products in your setup.

Or is there any way I can access you setup online, so I could check these myself?

Thanks for taking the time installing this.

Cheers,

Rob

cwdmark · July 24, 2003

where does it print out parse time and query time? maybe you have a newer version of osc than i do..

i'll post both catalog files once i know how to add those times in... i'd like to know myself too. this store im working with has about 5000 items online right now

July 24, 2003

where does it print out parse time and query time? maybe you have a newer version of osc than i do..

i'll post both catalog files once i know how to add those times in... i'd like to know myself too. this store im working with has about 5000 items online right now

did you download the second file that extracts to two directories named 'catalog_ori' and 'catalog_new' ?

cwdmark · July 24, 2003

no i didnt download anything... i'll do it later, im busy now

July 24, 2003

where does it print out parse time and query time?

I just realised:

Did you read this in the readme?

2. Make sure 'Display The Page Parse Time' is set to 'true' in admin ->

configuration -> logging.

It should display timing data on the bottom of the page

July 24, 2003

no i didnt download anything... i'll do it later, im busy now

Sorry, I misinterpreted your earlier post. I thought you meant you downloaded and tried my new category code.

Sorry about the confusion,

Rob

azer · July 25, 2003

why no team members give any sugestions ?

July 25, 2003

why no team members give any sugestions ?

I don't know exactly what you mean, but as I said the code is not ready to be considered including into osC.

I'm just trying to get some benchmarks, to see if I am on the right track with this approach.

If I get back some favourable benchmark results I'll finish it, tidy up the code and get some more testing done, after that I'll try and get it on the radar for the core team.

So far I do see some interest, I am logging some downloads of the code, but I have only one person promising to post some results to the thread. Which is kinda disappointing.

Cheers,

Rob

July 26, 2003

I owe everyone an apology

I am a moron, I am sorry

So there, with that out of the way: The last tarball some of you downloaded was doing everything it should do except reporting the benchmark figures, unless you turned on sql query logging. :oops:

To make up for this stupidity I have uploaded a new version of the code and have setup a demonstration server online.

So, for the downloads

The new code:

http://www.cryp2nite.nl/improved_catalog.tar.gz

Previous version:

http://www.cryp2nite.nl/improved_catalog-0.2.tar.gz

This code is the same (except for the SQL connection settings) as the demo links below.

Demo links:

Original code (Today's CVS)

http://www.cryp2nite.nl/oscdemo/catalog_ori

New code

http://www.cryp2nite.nl/oscdemo/catalog_new

For the performance improvement look at the reported, number of queries and the total time spent doing these queries.

Parse time is probably not a very useful, as I have paid no particular attention to performance for the rendering code and the generated page is showing too much information, but as you can see all required information to construct a decent menu is there.

I hope I can still convince some of you to give it another shot and try the code on their setups.

Cheers,

Rob

P.S.: For anyone not downloading the code, there's a README with some instructions on how to compare the two versions of the code. It might be useful because the category box of the new code is not functional and navigation is not possible, it's also avalable online:

http://www.cryp2nite.nl/oscdemo/README.TXT

Most should be pretty obvious though.

July 26, 2003

I forgot:

If this version does not work for you the way the demo links do, or you can't get it to work, just let me know and post to the thread.

Just tell me I messed up again, I'm a big boy I can take it.

Cheers,

Rob

July 29, 2003

your demo times look fairly accurate compared to the limited testing i have done so far

it is an amazing speed difference

now what will it take to make navigation work?

July 29, 2003

Thanks for taking the time to look at this.

your demo times look fairly accurate compared to the limited testing i have done so far

Great, I am/was hoping someone with a big store to give this code a try and report back.

Could you give me an idea what size store your setup has and maybe some info on your hardware config?

it is an amazing speed difference

That is kind of what I expected, I did some (really fast and probably not too accurate) tests on a stock install with maybe 9 extra categories and products nested three levels deep. And the performance gain went up to well over 60% on some pages :D

now what will it take to make navigation work?

Well, probably for me to finish the code. To do that I would have to be convinced someone is going use this.

But, as I said in an other thread it's of limited use to me as I am not expecting any performance issues for the sites I am likely to setup in the near future.

My thought is that it would have to be included in osC proper, as this is not a change in functionality but a dfiferent aproach for standard osC functonaity.

But I don't know what the right approach is. What I do know that I haven't been around long enough to earn enough brownie points to have my claim taken at face value :). That's why I am trying to get some favourable performance numbers, to make sure I am not wasting everybodies time. And then to ask someone in the core team to have a look at this thread and ask for their thoughts.

Cheers,

Rob

July 30, 2003

now what will it take to make navigation work?

Your in luck :)

I decided to give it one last try and finish some code to do the layout. It's not the most efficient way of doing it, but it works.

Seeing as I have logged over 50 downloads of the code, I take it there are people interested in this code. Still there are nearly no responses in this thread. The only feedback I get in private messages is along the lines of: "The menus look all messed up in your new code". So I hope this helps.

I am still looking for some people to do their own benchmarks with this code. Especially if you have a shop with a large number of categories and products I would be very interested in the results on your setup. If you decide to download this and cannot get it to work, let me know and I'll do what I can to makke it work.

Please let me know if anyone is interested in this code, because as I said before, I probably won't benefit from it my self all that much in the shops I am to set up any time soon.

If noone wants it, or thinks it's a bad idea to begin with, that's fine with me, I'll just give it a rest then.

As usual:

Code download

New version:

http://www.cryp2nite.nl/improved_catalog.tar.gz

Previous version:

http://www.cryp2nite.nl/improved_catalog-0.3.tar.gz

Demo links:

Original code:

http://www.cryp2nite.nl/oscdemo/catalog_ori

Improved code:

http://www.cryp2nite.nl/oscdemo/catalog_new

Benchmark scores at the bottom of each page, be sure to browse to some of the categories for greater improvements.

Have fun,

Rob

BIG (up to 45%) performance increase for categorybox.php

Recommended Posts

Guest

Guest

Guest

Guest

skarra

Guest

Daemonj

Guest

Daemonj

Guest

Guest

cwdmark

Guest

cwdmark

Guest

cwdmark

Guest

Guest

azer

Guest

Guest

Guest

Guest

Guest

Guest

Archived

Browse

Activity