Guest Posted January 26, 2003 Share Posted January 26, 2003 Hello, We could be working on a version of oscommerce which would need to support 4-10 million products. Does anyone have any feel for how oscommerce scales? How does the search speed keep up? How does the general speed keep up? What are the strengths and weaknesses? I guess the best way would be to test it, but I was hoping to perhaps find somebody out there with some experience. Also, we could pay someone to tweak it so it becomes scalable. We're going to need transaction support and rollback features at some point, and we'd be happy to pay for some quality work. Any thoughts out there on MYSQL 4.1 vs. ORACLE? If there are any lead developers who may be able to help out, we could definitely compensate them fairly. My hunch is that OSCOMMERCE, on a monster machine(s) and with some tweaking, could handle quite a lot. We could provide the monster machine... Best, Elliot Link to comment Share on other sites More sharing options...
Ramesh Posted January 26, 2003 Share Posted January 26, 2003 Hi there, That's a large scale COMMERCIAL undertaking . My current OSC store has 150 products , the SQL file is 500K so 300 products = 1MB for me. MYSQL would probably not be the database of choice. Something like firebird or PostgreSQL would be a better bet. You would also need to pay for professional 24/7 support available via phone/fax/email . Most databases assume you are going to be performing far more reads than writes. When you say 4-10 million products, is that unique products ? Or are you including variations ? For example a pair of Jeans in 5 sizes and 10 colours . Is that one product or 50 (5x10) in your example ? The speed of the database, depends on : The hardware The type of query number of concurrent users and many more factors. This sounds like a major project, it would be great if you could use oscommerce. But , in my opinion you will need some sort of major commercial product with years of development behind it. Anyone else ? Special Effects / 3d + Flash Link to comment Share on other sites More sharing options...
badjuju Posted January 27, 2003 Share Posted January 27, 2003 phew! 4-10 millions products I'm not saying OScommerce can't handle it, because it's just software. I'm a software architect and I work enterprise-scale ecommerce software. Your main concern is for anything of that size, you definitely need a high performance server - these could cost between $250,000 - $500,000. You also need to factor in redundancy - definitely start off with multiple webservers on less expensive machines. Database should be on a diff machine. There's a lot of information, but you can email me if you need more information. [email protected] Link to comment Share on other sites More sharing options...
badjuju Posted January 27, 2003 Share Posted January 27, 2003 One more thing - If you have Oracle as an option, I'd definitely go with that - a lot more powerful. But at this time, I believe MySQL is only supported DB server - I may be wrong. Link to comment Share on other sites More sharing options...
wizardsandwars Posted January 27, 2003 Share Posted January 27, 2003 You're going from 1 extrame to another. An absolutly free DBMS to the most expensive on on the market. I am a profressional DB programmer, that works in a Data Warehouse team of programmers, and we handle 100 different 70+ million plus row tables. We use Sybase. This DBMS does have it's drawbacks (can't turn logging off) but it's an inexpensive solution, and it can handle anything oracle can. Just doesn't have all of the bells and whistles. I love oracle, and I am an OCP, but unless you've got 750K to spend on your DBMS, I wouldn't reccoment it. ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help. Link to comment Share on other sites More sharing options...
Guest Posted January 27, 2003 Share Posted January 27, 2003 Why wouldn't mysql scale to handle millions of records? Granted some of the database functions may have to be re-written to support transactions, but would this be terribly difficult? At any rate, the one way to answer this is for us to try it--just seeing if anyone's tried anything similar. We'll be trying it at some point at http://vvgallery.org , to see if it could sclae to handle 10,000,000 pictures on a quad-xeom machine, or osmthing similar. I'm trying to grasp what exactly it is that mysql4.1 doesn't have that oracle does have, which would allow growth to tens of milliosn of records. Do I really need stored procedures and triggers? That is the primary thing that mysql 4.1 is missing it seems, which will be available in future releases (5.0). Here's a list of mysql features: http://www.mysql.com/products/mysql/ Could anyone please characterize the bottlenecks or breaking points you would see in scaling mysql to support millions of records in an application of oscommerce, such as that seen at http://vvallery.org ? Best, Elliot Link to comment Share on other sites More sharing options...
wizardsandwars Posted January 27, 2003 Share Posted January 27, 2003 Well, one thing for sure is that the indexing is going to be very important with that many rows. A couple of things that you are going to want that MySQL doesn't offer is bitwize indexing, and covering indexes. As far as I know, MySQL can only create indexes for one field per index. This is horribly ineffieient, especially for larger databases. Also, you are sure to want update prices and availablilty, fields and descriptions, as well as add and remove products. Since MySQL doesn't come with a autmatic bulk Net Change solution, this means that you are going to want to do a complete refesh of your product tables. Well each time you do this, you are going to have to drop your indexes before loaded all of that data, and then recreating the indexes. This can all be simple automated, but with that many records, and with having anly one field per index, you are talking about several hours of index build time. And this is preferable to the dozen of hours or so it would take to load the data through the indexes. Not to mention that for both of these methods, you will need to remove access to these tables, and take your website offline while you do this. You can do it with MySQL. I just wouldn't advise it. With that many records, you need a fully functioned DBMS, the cheapest of which would be Sybase. Hope this helps. ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help. Link to comment Share on other sites More sharing options...
wizardsandwars Posted January 27, 2003 Share Posted January 27, 2003 Another thing to consider, that I han't thought of before, is logging transactions. MySQL offers a sort of 'undo' action that enables you to rollback the last change you made to the database. Thius is all fine and dandy, however in order to do this, it writes to a logfile every change it makes. Unfortunatly, this can make "inserting", "deleting", or "updating" tables take a very, very long time. In order to work around "logging" transactions, each DBMS has it's own way of dealing with it. Oracle lets you turn logging off. Sybase lets you preform a form of "DDL", call a "SELECT INTO". Unfortunatly, MySQL doesn't offer a way to circumvent logging its transactions, so you are stuck with slower preformance on larger tables. ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help. Link to comment Share on other sites More sharing options...
Vivien Posted April 21, 2003 Share Posted April 21, 2003 I know this post is very old, but I would like to know for what exactly you would use transactions with oscommerce. I have just learned about transactions and stuff, but I am not quire sure how to use them in our "field", or where are the advantages here with our shops? MySQL4 with InnoDB table types seems to do the same as Oracle, in terms of speed. There is a benchmark on the following article: http://www.eweek.com/article2/0,3959,293,00.asp PS: Anybody tried mysql4 with oscommerce? Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.