| View previous topic :: View next topic |
| Author |
Message |
mdg
Joined: 17 Jan 2006
Posts: 20
|
 Posted: Tue Jan 24, 2006 7:38 pm Post subject: Alternative to FTP in wview |
 |
|
I've been experimenting with an alternative to the FTP system in wview. I went down this path for a couple reasons.
First, most FTP clients send the password in clear text. I'm a little paranoid about such things, so I wanted something that supports SSL because tnftp does not. My FTP server supports it, so why not use it!
I also noticed that the wviewftp.conf uses wildcards. This gives the potential of uploading files that haven't changed and don't need to be sent.
To use FTP over SSL, I found lftp 3.4 (http://lftp.yar.ru/) is excellent for this. It has many of the same functions as tnftp, especially the ability to use a URL from the command line. It also takes a -f option where you can specify a script to transfer files. (The commands being FTP commands.)
First, to set up SSL, I put this in my ~/.lftp/rc file:
| Code: |
set ftp:ssl-allow on
set ftp:ssl-force on
|
Then I created a commands file like this:
| Code: |
open -u USERNAME,PASSWORD ftp://ftpserver.com/weather
mput /var/wview/img/*.png
mput /var/wview/img/*.htm
mput /var/wview/img/*.html
mput /var/wview/img/*.xml
mput /var/wview/img/*.txt
exit
|
Then from cron, I set up the command:
| Code: |
/usr/bin/lftp -f ~/ftpcommands.txt
|
You'll see the mput commands are the same as what is in the wviewftp.conf file. Now to change that.
Using the find command, you can find the most recently changed files like this:
| Code: |
find /var/wview/img -mmin -10 -print
|
The -mmin -10 specifies to look for things that are newer than 10 minutes old. (I'm transferring every 10 minutes right now.)
So now I wanted to take that list and make an lftp command file out of it. So I used perl to do this. While I was in there, I might as well FTP the files. So here is how I do it all in one step:
| Code: |
#!/usr/bin/perl
open(FTPLIST,">/tmp/ftp.list") || die "can't open file for reading: $!";
print FTPLIST "open -u USERNAME,PASSWORD ftp://ftpserver.com/weather\n";
open(FILELIST,"/usr/bin/find /var/wview/img -mmin -10 -print |");
while (<FILELIST>) { print FTPLIST "mput $_"; }
print FTPLIST "exit\n";
system("/usr/bin/lftp -f /tmp/ftp.list");
unlink </tmp/ftp.list>;
|
Normally, 66 files would be transferred with the old list. With this script, I only transfer as little as 36.
I hope this is useful. Maybe Mark would like to take some of these ideas and implement them into the program. |
|
| Back to top |
|
 |
mteel
Joined: 30 Jun 2005
Posts: 435
Location: Collinsville, TX
|
| Posted: Wed Jan 25, 2006 1:39 pm Post subject: |
|
|
Why wouldn't you just use scp via wviewsshd if you are worried about security? FTP over SSL? scp is very fast and secure. That's why wviewsshd exists - as an alternative for security minded users.
Mark |
|
| Back to top |
|
|
mteel
Joined: 30 Jun 2005
Posts: 435
Location: Collinsville, TX
|
| Posted: Wed Jan 25, 2006 3:59 pm Post subject: |
|
|
By the way, scp has all of the "don't copy unchanged files" logic built-in - it is actually a synchronization operation.
Mark |
|
| Back to top |
|
|
NorwegianWould
Joined: 07 Jan 2006
Posts: 23
|
| Posted: Thu Jan 26, 2006 8:23 am Post subject: |
|
|
You could use rsync too. This copies the minumum of files, and for large files will only copy the parts that have changed.
Jeremy |
|
| Back to top |
|
|
mteel
Joined: 30 Jun 2005
Posts: 435
Location: Collinsville, TX
|
| Posted: Thu Jan 26, 2006 8:27 am Post subject: |
|
|
In fact that is what wviewsshd uses: rsync. scp and rsync are both utilities built on top of ssh, thus my error in quoting scp earlier...
My point is this: Why roll your own when wview already has a secure solution using ssh/rsync?
Mark |
|
| Back to top |
|
|
mdg
Joined: 17 Jan 2006
Posts: 20
|
| Posted: Thu Jan 26, 2006 5:46 pm Post subject: |
|
|
| Because my ISP doesn't give me shell access, so I can't use anything else. I'm familiar with scp, I use it all the time at work and I'd use it here if I could. (I do applaud its inclusion in the software!) |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
