Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

[Contribution] Website Backup - Create zipped copy of all site files


spooks

Recommended Posts

How many times have you modified your site but the mod went wrong and you wish you had backed up first, this makes creating that backup very easy with just a single click.

 

This contribution creates a zip file of your entire site (excluding the database) for immediate download to your computer so making creation of a site backup quick and easy with just a single click.

 

Your backup, cached and sessions folders will be excluded, others can be easily added.

 

Security:

The code will check the user is logged into admin at all stages, so only administrators are able to obtain and download the compressed site copy.

 

Two zipping methods are provided, File and Memory (streaming).

 

File Mode Zipping.

The file method first creates a temporary zipped copy of the website within your backup folder then immediately provides the download of that, the temporary zipped copy is then immediately deleted. This process uses very little memory so will always work as long as there is sufficient disk space to store the temporary file.

 

Memory Mode Zipping.

With this method the zipping all occurs within memory and at the same time the download is provided (streaming). Memory usage is still relatively low so should not prove an issue, however this requires at least PHP Version 5.1.2, if your server has less than that the memory mode zipping option will not appear.

 

Compatibility: For osC 2.3.x and below

 

 

Contribution Download Available at http://addons.oscommerce.com/info/8387

Edited by spooks

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

Hi Sam,

 

I installed this mod today and get this error:

 

Fatal error: Class 'ZipArchive' not found in blah blah blah........

 

I looked in the includes/classes/zipstream.php file and it has the class as class ZipStream {

 

I tried changing the ZipArchive to ZipStream in backup_website.php and then I got undefined function open.

 

Everything seems so straighforward so I'm not really sure where it went wrong... :(

Matt

Link to comment
Share on other sites

Hi,

 

You didn't do anything wrong and you don't necessarily need to change anything! :o

 

ZipArchive is a standard php class, I assumed it would be included as standard, obviously I was wrong in that :blush:

 

Ask your hosts to add the ZipArchive class to your php

 

ZipArchive is only used for file mode zipping, the ZipStream class being used for memory mode (streaming)

 

To prevent the error message, change line 83 of backup_website.php from

 

$mode = isset($_POST['mode']) ? $_POST['mode'] : '';

 

to

 

$mode = isset($_POST['mode']) ? $_POST['mode'] : '';
  if ($mode == 'file' && !class_exists('ZipArchive')) $mode = '';

Edited by spooks

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

@@mattjt83

 

What version of PHP are you using then? Is it too early for the streaming option?

 

I have been considering adding a gzip option which would be much less restrictive, though I was thinking people would be using upto date servers generally.

 

This works on all the servers my clients use but perhaps some hosts don't take that much trouble.

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

@@mattjt83

 

So the memory (streaming) option should work fine for you, does it not? or is there a reason you only want the file mode?

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

Sam, after clicking the "here" button in both the File Mode Zipping and the Memory Mode Zipping I receive the following errors:

Notice: Constant PREVNEXT_BUTTON_PREV already defined in /hosting/www/xxx/www/admin/includes/languages/czech.php on line 388 Notice: Constant PREVNEXT_BUTTON_NEXT already defined in /hosting/www/...

.../includes/languages/add_ccgvdc_english.php on line 32 Notice: Constant TEXT_VALID_CATEGORIES_NAME already defined in /hosting/www/xxx/www/admin/includes/languages/add_ccgvdc_english.php on line 33 Fatal error: Call to undefined function tep_href_link() in /hosting/www/xxx/www/admin/includes/languages/czech.php on line 440

Absinthe Original Liquor Store

Link to comment
Share on other sites

Hi,

 

When you click either button the code performs a limited startup routine which does not call application _top.php thats as the output is not to be a page but just the download

 

The code does have to call some standard osC routines as the users status must be checked and some variable settings are needed.

 

The code calls languages/$language.php to get date formatting, it would appear your site or your browser is set to czech and that in turn is calling add_ccgvdc_english.php which seems to be doing some very non-standard stuff including adding links with tep_href_link

 

I would suspect you always get the notices, you just see them here as there is no page display.

 

The call to tep_href_link fails as my code doesn't call DIR_WS_FUNCTIONS . 'html_output.php' as nothing in that is needed

 

what is happening with your language files and what made them that way?

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

Not sure what is happening to the language files - the czech language is there as the shop runs in the Czech Republic and the whole language pack has been added as usual - no other errors in any other contributions has been detected or seen. The previous version Backup Store Files has been working fine as well. Same errors do appear if I switch admin language to English.

Absinthe Original Liquor Store

Link to comment
Share on other sites

Well clearly there is something odd that language pack, you wouldn't see the error generally as most add-ons will call application-top always, so instantiate DIR_WS_FUNCTIONS . 'html_output.php');

 

You could modify this to allow the vagaries of your files, I see no point in modifying the contribution itself.

 

To stop the error, in backup_website.php:

 

after(90):

 

require(DIR_WS_FUNCTIONS . 'general.php');

 

add:

 

require(DIR_WS_FUNCTIONS . 'html_output.php');

 

and remove that same line from line 170 (appears there purely of error display)

 

To stop the Notice Display caused by conflicts within your language files

 

after(10):

 

$guess = 0; // ...

 

add:

 

error_reporting(E_ALL & ~E_NOTICE);

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

The solution above worked, however none of the .zip files could be xtracted, receiving errors in Directory Opus as well as in WinRAR - ! C:\Users\User\Downloads\Website-Backup-26-03-2012 (1).zip: The archive is either in unknown format or damaged.

Absinthe Original Liquor Store

Link to comment
Share on other sites

@mr_absinthe

 

Was that with the memory (streaming) method, it states in the doc that can happen if there size estimate is too low, try increasing the guess value. For the file method it uses 2% over final file size but I believe in rare cases that may be insufficient, the problem is with zipped content the download stops the moment content_length is reached, if thats a single byte short of the full file it will be seen as corrupt.

 

Alternatively you can take size out of the equation by removing from ZipStream.php the line (521):

 

'Content-Length'			=> $this->size

 

for streaming & for file mode from backup_website.php the line(161)

 

header("Content-Length: ".floor(filesize(DIR_FS_BACKUP . $filename . $filedate)*1.02));

 

Though that will make the download less friendly.

 

@@mattjt83

 

Could you say what platform your using (windows/linux) and if you could take a look in your apache error log to see if it sheds any light on what went wrong.

 

I must admit I haven't tested on a windows server but I thought I had caught all the bugs in ZipStream.php

Edited by spooks

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

I've increased $guess = 1; and now receiving this error when I use both File Mode Zipping and Memory Mode Zipping:

 

Warning: array_map(): Argument #2 should be an array in /hosting/www/xxx/www/admin/backup_website.php on line 121 plus many headers already sent warnings.

When I changed it to $guess = 2; - file downloaded but unable to open - corrupted. (Estimated compressed file size: 18.22 MB - real size 12 MB - quite a difference)

When changed to $guess = 3; - same as $guess = 1;

Edited by mr_absinthe

Absinthe Original Liquor Store

Link to comment
Share on other sites

@mr_absinthe

 

Is your backup folder empty by any chance? Sorry, not something I thought of :blush:

 

change the line to

@array_map( "unlink", glob( DIR_FS_BACKUP . $filename."*" ) );

 

i.e. just add the @ in front.

 

Your issue isn't the size, its content sent before the header, any warnings/notices etc will cause that, ensure you have the code I posted before to stop the notices

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

I've added the code as instructed:


  $exclude = 'DIR_FS_BACKUP, DIR_FS_CACHE, SESSION_WRITE_DIRECTORY, contest/'; // add excluded as required, include full path from catalog folder i.e. includes/languages/
  ini_set('memory_limit', '128M'); // set memory limit, increase if you receive an out of memory error, you may need to speak to your host!!
  $guess = 4; // increase if size estimation is too low
  error_reporting(E_ALL & ~E_NOTICE);

 

The backup folder is not empty, but I've added the @ anyway:

@array_map( "unlink", glob( DIR_FS_BACKUP . $filename."*" ) );

 

$guess = 1; - corrupted file

$guess = 2; - corrupted file

$guess = 3; - corrupted file

$guess = 4; - corrupted file

 

Tested with both backup modes.The good news is that there are no errors, at least visible ^_^

Absinthe Original Liquor Store

Link to comment
Share on other sites

@mr_absinthe

 

Its probably something in the headers your server don't like (could you say if windows or Linux) could you check the file creation is good, comment out line 169

@unlink(DIR_FS_BACKUP . $filename . $filedate);

then run file mode, then take a look in your backup folder, you will find the zipped file still there, if that's OK then its something in the download process thats causing it.

 

You can't test in memory mode, as no file is stored.

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

It is Linux, php 5.3.10.

$guess = 1;

File on the server is OK and the file that was downloaded is corrupted.

Estimated compressed file size: 17.96 MB - real size 12MB. So I played with $guess just for fun and the most realistic estimated was achieved with value $guess = -20;.

I've made no changes to ZipStream.php BTW.

Edited by mr_absinthe

Absinthe Original Liquor Store

Link to comment
Share on other sites

Hi Spooks,

 

I've just installed this great contribuiton (thanks for providing it), but I'm having some strange behaviors.

I get the following statistics:

Website size: 24.16 MB Estimated compressed file size: 16.43 MB Containing 257 folders with 6467 Files.

 

I tested with both zipping methods with the following results (I have added other excluded folders):

  • Testing with File Mode Zipping
    • gives a 500 error in the server. Looking at the logs I see the error
      [/list]
      Premature end of script headers: backup_website.php
      

    • When I look at the backup folder, I was expecting to see there the Zip file, or a part of it at least, but nothing, I just have there the database backup files. Even while the backup is running, I look at the folder and I see nothing there. Is it expected?

  • Testing with Memory Mode Zipping (same configuration as for the other test):
    • I'm able to download the ZIP file, but as the estimated compressed file is 16.43MB and the real file is 11.1MB, the download runs well until it reaches the real file size, then it seems to be stopped for some time (I did not count the time) and then suddenly it says the download was complete. The file is ok, and extracts correctly.
    • Then, regarding the statistics I see:
      • The website size is more or less correct
      • the estimated compressed size is way too big (the downloaded file is 11.1MB or 11,677,866 bytes)
      • the file count is completely wrong, I get 2450 files, pus the Backup info.txt you add. I've done a "manual" backup to compare with, by using a complete copy of the site and deleting the excluded folders and I got 2458 files: all files with 0 bytes are skipped. All empty folders are also skipped. Is it normal?

I'm running on a shared server, under linux with PHP 5.2.14, and the guess varialble is set to 0.

Link to comment
Share on other sites

I`ll modify this for more widely compatible headers, as that seems to be a general issue.

 

@@modem2.0 You are unlikely to see the file as its not there very long, its deleted the moment the download data is flushed. You have to disable the delete to catch it.

 

Memory Mode Zipping: The file estimate is very approximate its accuracy depends on how many images you have, as they take very little or no compression. It must be larger as explained in the doc.

 

That 'pause' caused by the size discrepancy. I may look at removing size, but its handy for large sites.

 

Empty folders are an issue as is anything with a zero size, I`ll see if that can be improved. The file count is just for info & not used in processing, I'll check if its missing though.

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

Thanks Spooks.

Regarding your comment:

@@modem2.0 You are unlikely to see the file as its not there very long, its deleted the moment the download data is flushed. You have to disable the delete to catch it.

It takes around 45 seconds to get the 500 error, so during this time I refreshed the Backup FTP folder several times and I never see the file being created, anyway I'm not even sure I should see it. But tomorrow I'll disable the delete and test it again.

The download never starts, I'm assuming the error comes during the ZIP file creation.

 

I'm surprised for having the memory mode working and the file mode not, as I had the old contribution but since more than 6 months ago the backups stopped working (I have added some contributions with lot's of files, and maybe the algorithm is less efficient than yours).

 

Regarding the images, in my case, I excluded the products images, that are taking around 200MB.

 

Best regards

Link to comment
Share on other sites

@@modem2.0

 

The old contrib was very heavy on memory usage & would fail if a site was too big

 

The two modes work quite differently, the file mode uses the ZipArchive class that's a standard php class but may not exist on all servers, its very efficient with memory but the file has to be created on disk.

 

The memory mode uses a completely separate custom streaming class

 

The next version will also have a GZipping option, the reason for all the different options is there are problems with each and some cannot run on some servers, so by having a choice, at least one should work.

 

Check you have sufficient disk space and its not a permissions issue. I cannot guarantee there's not an issue with the ZipArchive class, although is a standard class it could still have bugs, ie the standard pear tar class is bugged.

Edited by spooks

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

Hi Spooks,

 

I already tried the file mode without deleting the generated ZIP file, and I still cannot see it, maybe I cannot see it before the file is completely created...

I tried also excluding some more folders: admin, javascript, and phpThumb to reduce the number of files and the size of the backup just for the test, and still get the 500 error.

 

I have more than 200MB free on the server, and I'm backing up just 24MB (without the excludes above), so it seems it's not a disk space issue.

 

Regarding permissions... Well, I can backup the database successfully to the Backups folder, so for this one should be the same, I guess...

 

Is there a way to see if the ZipArchive class is installed?

 

Best regards

Link to comment
Share on other sites

@@modem2.0

 

The next version includes a test for the class (out very soon) but if you have another class of the same named defined already (from some other add-on) that could cause problems

 

That's one issue namespaces (used in osC3) is to avoid

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...