Uncompress on host

[drakonan]

Someone had bought up the idea of uploading a large zip file (or other compressed file) to his host, and unzipping it on the host.

 

It'd be a fast way of uploading. For me esp. since I have satellite. It's high speed, but since the latency is so high, doing a single process/thread file transfer (such as in FTP) it's potentially even worse than dialup. (IE a larger 1 meg file could take as long as a 1kb file, with all the negotiations etc involved)

 

Anyway, to the point:

 

I use Windows, the host uses some form of Linux.

 

Is it possible to compress the file, upload it, and uncompress it? (on these different platforms?)

[Emmtee]

Someone had bought up the idea of uploading a large zip file (or other compressed file) to his host, and unzipping it on the host.

 

It'd be a fast way of uploading. For me esp. since I have satellite. It's high speed, but since the latency is so high, doing a single process/thread file transfer (such as in FTP) it's potentially even worse than dialup. (IE a larger 1 meg file could take as long as a 1kb file, with all the negotiations etc involved)

 

Anyway, to the point:

 

I use Windows, the host uses some form of Linux.

 

Is it possible to compress the file, upload it, and uncompress it? (on these different platforms?)

 

I use rsync and a small alias to call it as "upsync" or "downsync" ... it can keep directories on the server in perfect synchronisation while beeing very fast :)

[drakonan]

rsync will only create directories one deep.

 

Is that true?

 

Since I use osCommerce, I definitely go beyond "one deep."

 

Is this something I install on my windows box?

 

Anyway, i'm interested.

 

I can't get my osC site online until I figure this out. (My satellite, is VERY unreliable, after begining to upload multiple files, then eventually

somehow the there's a transmission error of some sort, the first character of the ftp client's command sometimes disappears...

 

And at any rate, the ftp server says what are you talking about?

"ORT not understood" (see the P of port is missing) or "Tor not understood" (see the S on stor is missing)

 

The client has no clue what to do and just disconnects...

[Emmtee]

not true

 

upload to server (keep files on server that do not exist on local dir)

rsync -e ssh -avzp --delete /localwebdir/shop/ server:/WWWhome/shop/

 

download from server (delete local files not exisiting on server):

rsync -e ssh -avzp --delete server:/WWWhome/shop/* /localwebdir/shop/

 

note: i use ssh and configured hostkey-authentication, placed those 2 commands into files called upsync and downsync (placed correct chmod) and thus all i have to type is "upsync" or "downsync" ... the rest will be mirrored even up to directory permissions...

 

the rsync manpage is actually quite nice, take a look, it has lots of additional functions :)

[Emmtee]

ps: tcp connections usually should NEVER have transmission errors as the connection data is checked... that problem is probably an error with your TCP (sat) stack or the ftp client (or server)...

 

i know about rsync beeing able to even PATCH files - look for differences and just transmit those.... needless to say it can resume files... but use the corresponding switches :)

[drakonan]

Thanks for your effort in helping me. This does seem like a powerful utility. I'll finish setting it up tomorrow... It's 1 o' clock in the morn and it's been a very very long day :)

 

Thanks again.

 

I like the idea of just transmitting the changes. How do you set this up so it works through "SSH"?

 

I guess by default it tries to use its own or somthing "RSH"?

[Emmtee]

How do you set this up so it works through "SSH"?

I guess by default it tries to use its own or somthing "RSH"?

obviously it's late :)

-e SSH in command call - rsync is enabled to use ssh in a transparent way :)

[drakonan]

By the way... about the error... it's always only the first char... IF it was same satellite error I'd think it'd be random chars...

 

The missing first letter almost sounds like there's a compatibility issues with maybe the high latency ftp server and client connection.

What do you use for an FTP client? I'm using Smart FTP.

[Emmtee]

i know only 4 stable FTP tools (and i tested a LOT of them):

 

ftp.exe (commandline, supplied with OS)

ncftp

ws_ftp

flashFXP

 

select your weapon of choice :)

... nontheless rsync will work MUCH faster and with LESS headache when used to sync a complicated directory tree... that's an weakpoint of FTP (not to mention FTP is NOT encrypted)... in short: rsync is weapon of choice :)

 

I even know a friend of mine who's using rsync on windows ...

[drakonan]

That's the issue. I don't know linux even well enough to use rsync under windows!

 

It sounds tantalizing though.

[drakonan]

After hours of searching and attempting to install rsync inside cygwin, I finally read that rsync can be installed with it automatically!!!!

 

Well... I did that and "cygpop-0.dll" is missing... Ok, great! What is this file, I wonder... As a last ditch attempt I install all the "by default" downloaded programs and it works.

 

NOW, I can't find the file to configure so it know WHERE my server is and what my login is. I thought it was under a package dir, but I can't find one.

 

Now I know why Windows is used by more end users!!!

 

Linux is vastly more complicated than any DOS ever was.

[drakonan]

Another Q. Does my host need rsync installed as well?

 

This is a great idea and anyone using satellite could substantially benefit from the program... I'm going to write

 

"Rsync on Windows for Dumbies" and make millions...

[Emmtee]

After hours of searching and attempting to install rsync inside cygwin, I finally read that rsync can be installed with it automatically!!!!

 

Well... I did that and "cygpop-0.dll" is missing... Ok, great! What is this file, I wonder... As a last ditch attempt I install all the "by default" downloaded programs and it works.

 

NOW, I can't find the file to configure so it know WHERE my server is and what my login is. I thought it was under a package dir, but I can't find one.

 

Now I know why Windows is used by more end users!!!

 

Linux is vastly more complicated than any DOS ever was.

 

well, rsync needs 1-2 cygwin dlls, openssl binaries and there you go after setting your HOME environment-variable.. (read manual and readme page)

 

FYI - READ THE COMMAND LINE EXAMPLES THAT I POSTET AGAIN

rsync is a unix/linux tool and designed to be used by many other tools (all major unix tools support the concept of chainability/piping/reusability): they provide a single function, but they provide it properly.

that's why you need to write a .bat file if you want to call rsync without need to specify all parameters again...

 

hint: test if you can login with ssl to your host, if that works, then rsync should work too - usually ssh is more tricky to setup than rsync.

 

if your server is a windows-box, then you're probably facing some setup on serverside.... just login via ssh to your server and enter "uname -a"...

or try if rsync exists remotely ... if yes, then it should work out of the box.

[Emmtee]

argh, i should use the preview function more often, so many spelling-errors :)

[drakonan]

Thanks again for your troubles. I think I've got it down now... MY host already had it installed. They charge $25 a month, but they have always had what I needed, before I needed it, as in this case rsync.

 

(and even unzip)

 

I'm learning so much... I love it.

 

Thanks again.

[drakonan]

Thanks for the help, I finally got it working, but have no clue how to create "downsync" for "rsync -options" and upsync for "rsync -options"

 

Could you teach me a little about the Linux world?

 

I thought maybe alias would do it, but I guess I'm either wrong or not doing it right.

 

BTW: When I responded to you the other time, I was reading an inrefreshed post, and so didn't realize that you had responded LOL.