Backup All Store Files In Zip Format

[spooks]

Backup All Store Files In Zip Format

 

 

This contribution will allow you to save a backup of all your site files in compressed (zip) format on your computer. It will ignore all files in your backup folder as well as your cached ones.

 

My main reason for this release is to address a security hole that was opened by the original contribution.

 

 

Features (v1.1):

 

1.Properly incorporated within admin, you must be logged in to back up.

2.Backup directory is excluded (save your dBase backups separately).

3.The cache folder is ignored (as defined within admin).

4.Will work under php 4.

5.Protected commands (exec etc) are not used.

 

Tested on PHP 4 & 5, SQL 4 & 5, osC 2.2 ms2, rc1 & rc2a and is register_globals off compatible.

 

 

Contribution is found here: http://addons.oscommerce.com/info/6986

 

 

[spooks]

Some Tips on use.

 

1. The average size of backups is 4-6Mb, if your is much bigger it may be a sign that your site has become some what bloated, check for unused files.

 

 

Remove Unused Images http://addons.oscommerce.com/info/6346 is useful at finding pictures u no longer use.

 

 

2. If on clicking on 'Click to save a copy of all store files' results in a blank page its most likely you ran out of memory, increase the value on line 19 of backup_store_files.php

 

ini_set('memory_limit', '32M');

 

change the 32 to 64 or 128. Though you may have to speak to your host b4 this will work.

 

 

3. On some servers the file you upload is double zipped, if this is so, on unzipping you will just get a file with no extension, add .zip to that filename, then u can unzip your site files as normal.

 

 

With luck, you will never need your backup, but everyone can't always be lucky!!

[francois21]

hello

excuse my bad english

all is good,

I put the value memory

ini_set('memory_limit', '128M');

but when I click I have this message

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 46277628 bytes) in C:\wamp\www\bouticsystem\admin\includes\classes\zip.class.php on line 178

 

I am in local

an idea?

Thank's

[spooks]

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 46277628 bytes) in C:\wamp\www\bouticsystem\admin\includes\classes\zip.class.php on line 178

 

I am in local

 

 

 

If your in local you should have no issue increasing further, try 256, 512 etc. But I would have to wonder how come u have so many files, perhaps you have too many large pictures, remember u must keep images to a reasonable limit for the web, 800 x 600 at average jpg compression is my recomendation.

 

The current version places all data in memory b4 compressing, I may look at other ways if it is a big issue.

[♥Biancoblu]

probably a stupid question, but are we meant to add a link or something in admin/includes/boxes/tools.php?

because I uploaded your package as is and I can't see where to click to save the files. :blush:

 

edited to add:

forget it, I didn't read the second page of your word document....sorry bout that. :blush:

Edited October 11, 2009 by Biancoblu

[pedro.antunes]

Hi Spooks,

 

I got the same error as Postfrancois21, then I tried to increse memory size to 256MB and the error is now "Allowed memory size of 268435456 bytes exhausted (tried to allocate 87165787 bytes)". I don't why it takes so much memory on the server since the copy of my site on my HDD is around 90MB including some DB backups.

Since the biggest part of my site is made of product pictures (+2000 product images + thumbnails), I wonder if there is a way of bypassing a set of folders or file extensions... When bypassing the images folders, 32MB should be more that enough...

 

I've done a quick look on the code and I could not find where the backups and cache folders are bypassed. It is also mentioned on the documentation that "The cache folder is ignored (as defined within admin)." but I was not able to find where it is defined...

 

I tried to change the code a little bit to bypass some folders:

on the function I changed from

	
case "dir":
 $file = str_replace("//", "/", "$file/");
 $return = lista_arquivos($file, $tipo, $return);

 

to

case "dir":
 if ($file !== DIR_FS_ADMIN."backups" and $file !== DIR_FS_DOCUMENT_ROOT."images" and $file !== DIR_FS_DOCUMENT_ROOT."tmp") {
   $file = str_replace("//", "/", "$file/");
   $return = lista_arquivos($file, $tipo, $return);
 }

 

Now I don't have the memory error but I have a different error:

"The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression."

Ideas?

 

Br,

Pedro

[spooks]

 

 

The memory requirements are higher than should be and I`ve not studied the whole code (the original is not mine) but it does need to hold a full copy of the site + compressed version in memory, so it will always be somewhat more than just the site size, currently.

 

 

The line that bypasses the dirs is 59:

 

if (strpos($arquivo,DIR_FS_BACKUP) === 0 || strpos($arquivo,DIR_FS_CACHE) === 0) continue;

 

to add images to that set, change it to:

 

if (strpos($arquivo,DIR_FS_BACKUP) === 0 || strpos($arquivo,DIR_FS_CACHE) === 0 || strpos($arquivo,DIR_FS_CATALOG . DIR_WS_IMAGES) === 0) continue;

[pedro.antunes]

Problem solved! The problem was due to some echo I left for debugging...

 

I also changed the filename creation from:

$filename = substr(STORE_NAME,0,12). '_SiteBackup-'. $current_date . '.zip';

to:

$filename = str_replace(" ", "_", STORE_NAME). '_SiteBackup-'. str_replace("/", "-",$current_date) . '.zip';

Since my store name contains spaces, when firefox asks for the name and folder to save the file it was just writing the 1st name of the shop and without file extension, and the saved file had the extention .gz, which fails to open in WinZip unless you rename it to ZIP first.

The change on the date was because my server returns the date in the format dd/mm/yyyy.

 

And I confirm, with the code I posted above, the folders backups under admin and images and tmp from catalog are not included in the ZIP file as expected.

 

Now my question:

All my files are stored on the server in:

/var/www/vhosts/my_server.com/httpdocs/

but in the zip it comes from:

r/www/vhosts/my_server.com/httpdocs/

But is it possible that in the ZIP not having the full var/www/vhosts/my_server.com/httpdocs/ path?

 

Regards

[pedro.antunes]

The line that bypasses the dirs is 59:

 

if (strpos($arquivo,DIR_FS_BACKUP) === 0 || strpos($arquivo,DIR_FS_CACHE) === 0) continue;

 

to add images to that set, change it to:

 

if (strpos($arquivo,DIR_FS_BACKUP) === 0 || strpos($arquivo,DIR_FS_CACHE) === 0 || strpos($arquivo,DIR_FS_CATALOG . DIR_WS_IMAGES) === 0) continue;

 

I'm blind... I did not see this line so I duplicated them on the function lista_arquivos...

 

Thanks!

Edited October 15, 2009 by pedro.antunes

[pedro.antunes]

Now my question:

All my files are stored on the server in:

/var/www/vhosts/my_server.com/httpdocs/

but in the zip it comes from:

r/www/vhosts/my_server.com/httpdocs/

But is it possible that in the ZIP not having the full var/www/vhosts/my_server.com/httpdocs/ path?

 

Regards

 

Found the solution to both questions...

the code

$nome=substr($arquivo, 3);

is "eating" the 3 first letters from the file: /va... from /var/www/...

changing the 3 to 1 makes the 1st folder name complete var/www/...

 

By changing that line to

$nome=str_replace(DIR_FS_DOCUMENT_ROOT,'',$arquivo);

removes the full path /var/www/vhosts/my_server.com/httpdocs/

 

Regards!

[spooks]

Full path not in zip

 

Your str_replace additions are a good idea.

 

edit,

 

see u posted just as I was, well done finding the problem.

Edited October 15, 2009 by spooks

[pedro.antunes]

Last one, I guess...

 

I used your method to remove the folders

  if (strpos($arquivo,DIR_FS_BACKUP) === 0 || strpos($arquivo,DIR_FS_CACHE) === 0 || strpos($arquivo,DIR_FS_CATALOG . DIR_WS_IMAGES) === 0) continue;

but the cache folders (from admin and from catalog) are still in the backup file.

The if needs to be updated to:

  if (strpos($arquivo,DIR_FS_BACKUP) === 0 || strpos($arquivo,DIR_FS_ADMIN . DIR_FS_CACHE) === 0 || strpos($arquivo,DIR_FS_CATALOG . DIR_FS_CACHE) === 0 || strpos($arquivo,DIR_FS_CATALOG . DIR_WS_IMAGES) === 0) continue;

to remove cache folders from admin and catalog.

 

Regards.

[spooks]

Last one, I guess...

 

I used your method to remove the folders

  if (strpos($arquivo,DIR_FS_BACKUP) === 0 || strpos($arquivo,DIR_FS_CACHE) === 0 || strpos($arquivo,DIR_FS_CATALOG . DIR_WS_IMAGES) === 0) continue;

but the cache folders (from admin and from catalog) are still in the backup file.

The if needs to be updated to:

  if (strpos($arquivo,DIR_FS_BACKUP) === 0 || strpos($arquivo,DIR_FS_ADMIN . DIR_FS_CACHE) === 0 || strpos($arquivo,DIR_FS_CATALOG . DIR_FS_CACHE) === 0 || strpos($arquivo,DIR_FS_CATALOG . DIR_WS_IMAGES) === 0) continue;

to remove cache folders from admin and catalog.

 

Regards.

 

Thanks, that would depend on the contents of DIR_FS_CACHE, although the default is only the name of the cache folder, this often causes problems, ideally DIR_FS_CACHE should contain the full path to the folder, so in any update, my preferance in this case would be to simply advise of the prefered contents of DIR_FS_CACHE.

[Guest]

hi! I'm new here and I just installed this contrib. The problem I have right now is that when I tried to test it out, after clicking the Click Here button to save the file as a zip, I am taken to an error 500 page. I'm trying to figure out why, but I'm not really sure. Can someone help me?

[spooks]

error 500 page

.

 

 

Check you htaccess in your admin directory, thats what commonly causes such errors.

[Guest]

error 500 page

 

 

Check you htaccess in your admin directory, thats what commonly causes such errors.

 

Ok thanks! what code, if any, should I put in there, or change?

 

--Edit--

 

Also, is there anything I need to do about folder permissions, for example?

Edited October 20, 2009 by westindiestrader

[spooks]

Ok thanks! what code, if any, should I put in there, or change?

Also, is there anything I need to do about folder permissions, for example?

 

 

U should post the code here from the file (with all passwords removed) so i can see what may be causing the issue.

 

As the app only reads the data, No folder permissions should need altering.

[Guest]

U should post the code here from the file (with all passwords removed) so i can see what may be causing the issue.

 

As the app only reads the data, No folder permissions should need altering.

 

Ok here it is (where passwords are concerned, it seems to only refer to the .htpasswd doc, as far as I can tell):

 

# $Id: .htaccess,v 1.1.1.1 2005/08/16 08:31:30 sstolyarov Exp $

#

# This is used with Apache WebServers

#

# For this to work, you must include the parameter 'Options' to

# the AllowOverride configuration

#

# Example:

#

# <Directory "/usr/local/apache/htdocs">

# AllowOverride Options

# </Directory>

#

# 'All' with also work. (This configuration is in the

# apache/conf/httpd.conf file)

 

# The following makes adjustments to the SSL protocol for Internet

# Explorer browsers

 

<IfModule mod_setenvif.c>

<IfDefine SSL>

SetEnvIf User-Agent ".*MSIE.*" \

nokeepalive ssl-unclean-shutdown \

downgrade-1.0 force-response-1.0

</IfDefine>

</IfModule>

 

# Fix certain PHP values

 

#<IfModule mod_php4.c>

# php_value session.use_trans_sid 0

# php_value register_globals 1

#</IfModule>

 

AuthUserFile /var/www/vhosts/westindiestrader.com/httpdocs/admin/.htpasswd

AuthName Secure

AuthType Basic

require valid-user

[spooks]

 

 

I can't see anything there, but just in case, can u try temporarily removing the file & see if that has an effect.

 

Also is your admin on a secure server?

[Guest]

I can't see anything there, but just in case, can u try temporarily removing the file & see if that has an effect.

 

Also is your admin on a secure server?

 

 

Good question. I believe it's secure. Permissions for the admin directory are at 755 and it is ssl enabled, if that's what you mean.

[Guest]

Hi spooks

 

My images folders have a lot of images in them, (thousands of products).

Increasing the memory to 1024 still results in blank page.

Can you provide a solution that doesnt place all files to memory b4 compressing?

 

Windows

Edited October 22, 2009 by Windows

[spooks]

Hi spooks

 

My images folders have a lot of images in them, (thousands of products).

Increasing the memory to 1024 still results in blank page.

Can you provide a solution that doesnt place all files to memory b4 compressing?

 

Windows

 

 

I plan to look at some point, but it wont be soon, sorry

 

look ealier here to exlude the images folder, that might do for now.

[Mazinfriki]

Hi,

 

I've excluded the images directory as said before and now it works for me. Thank you for this nice contrib :)

 

Also I would suggest you and option to store the zip file on backups dir instead downloading as database backup does, this would be also very useful.

[blurredreality]

Hey there this seems like a really good add-on - just having a few problems getting it going..

 

First of all I got the fatal error mentioned on page one - with the memory exhausted. I thought this was odd as I didn't particularly think I had that big a store section - but obviously I did!

 

I then changed the backup_store_files.php as recommended to exclude the image folder - and indeed that worked in that I did not get taken to the Fatal Error page.

 

Howevr, a file download box opened and then before any progress was made an error box popped up saying:

 

"Internet Explorer cannot download backup_store_files.php from securexx.inmotionhosting.com. It is unable to open this Internet site - the required site is either unavailable or cannot be found. Please try again later"

 

I have tried a few times and keep getting this.

 

Hopefully it's something small that I've missed!

 

 

-To clarify, my site has a shared ssl certificate, and the admin area has ssl turned on - and otherwise seems to be working fine. Just wondered if that may be the problem though?

Edited November 28, 2009 by blurredreality

[spooks]

Howevr, a file download box opened and then before any progress was made an error box popped up saying:

 

"Internet Explorer cannot download backup_store_files.php from securexx.inmotionhosting.com. It is unable to open this Internet site - the required site is either unavailable or cannot be found. Please try again later"

 

 

 

-To clarify, my site has a shared ssl certificate, and the admin area has ssl turned on - and otherwise seems to be working fine. Just wondered if that may be the problem though?

 

Hi,

 

I must admit I hadn't tested in that mode, but now I have & can confirm that this is indeed a issue, When I've time I will see if I can find a solution, in the meantime you could simply turn off your ssl for admin do the backup, then revert, its a lot faster than downloading it all by ftp still.