10

First of: I know of pyinotify.

What I want is an upload service to my home server using Dropbox.

I will have a Dropbox's shared folder on my home server. Everytime someone else, who is sharing that folder, puts anything into that folder, I want my home server to wait until it is fully uploaded and move all the files to another folder, and removing those files from the Dropbox folder, thus, saving Dropbox space.

The thing here is, I can't just track for changes in the folder and move the files right away, because if someone uploads a large file, Dropbox will already start downloading and therefore showing changes in the folder on my home server.

Is there some workaround? Is that somehow possible with the Dropbox API?

Haven't tried it myself, but the Dropbox CLI version seems to have a 'filestatus' method to check for current file status. Will report back when I have tried it myself.

cherrun
  • 1,862
  • 8
  • 30
  • 50
  • What happens if an attempt is made to move an incomplete file? – martineau Sep 09 '12 at 14:38
  • That's a good question. I haven't tried. IIRC you just ended up with a corrupted file. No warnings. – cherrun Sep 09 '12 at 16:33
  • Too bad, because if it was something unique and/or relatively easy to detect, then you could just delay (ie sleep) a while and retry. – martineau Sep 09 '12 at 16:44
  • You might be able to use (or convert to the OS of your choice) [A Python script to get the Dropbox status of a file or folder in Windows](http://www.dropboxwiki.com/Python_Script_To_Get_File_Or_Folder_Status_In_Windows). – martineau Sep 09 '12 at 17:11
  • Nice script, however, only available with Win32 libs or in Delphi. However, this got me in the right direction. Didn't know that there was a CLI Dropbox version. There is a function called filestatus. Will try that out. http://www.dropboxwiki.com/Using_Dropbox_CLI – cherrun Sep 09 '12 at 21:50
  • 1
    Dropbox CLI seems to be the way to go. You should post the link you've got here, and write up an answer to your own question. – Droogans Sep 09 '12 at 22:59

3 Answers3

2

There is a Python dropbox CLI client, as you mentioned in your question. It returns "Idle..." when it isn't actively processing files. The absolutely simplest mechanism I can imagine for achieving what you want would be a while loop that checked the output of dropbox.py filestatus /home/directory/to/watch and performed an scp of the contents and then a delete on the contents if that suceeded. Then slept for five minutes or so.

Something like:

import time
from subprocess import check_call, check_output
DIR = "/directory/to/watch/"
REMOTE_DIR = "user@my_server.com:/folder"

While True:
    if check_output(["dropbox.py", "status", DIR]) == "\nIdle...":
        if check_call(["scp", "-r", DIR + "*", REMOTE_DIR]):
            check_call(["rm", "-rf", DIR + "*"])
    time.sleep(360)

Of course I would be very careful when testing something like this, put the wrong thing in that second check_call and you could lose your filesystem.

aychedee
  • 22,113
  • 7
  • 74
  • 81
  • If you're going to use Unix command-line utilities from Python to delete the folder contents, you can also `chroot()` the utility so that "accidentally deleting your filesystem" never happens. – André Caron Sep 11 '12 at 22:20
  • Indeed, I potentially would prefer to do an os.walk of the directory and delete files that way. But this just seemed the easiest way to express the idea. – aychedee Sep 12 '12 at 07:16
  • Your script is not fully correct. `filestatus` never returns `Idle`, only `status` does. `filestatus` will only tell you if it is `up to date`. Well, I got the idea though. I might post up my own answer in the future. – cherrun Sep 12 '12 at 08:41
1

You could run incrond and have it wait for IN_CLOSE_WRITE events in your Dropbox folder. Then it would only be triggered when a file transfer completed.

user1664174
  • 11
  • 1
1

Here is a Ruby version that doesn't wait for Dropbox to be idle, therefore can actually start moving files, while it is still syncing. Also it ignores . and ... It actually checks the filestatus of each file within a given directory.

Then I would run this script either as a cronjob or in a separate screen.

directory = "path/to/dir"
destination = "location/to/move/to"

Dir.foreach(directory) do |item|
    next if item == '.' or item == '..'
    fileStatus = `~/bin/dropbox.py filestatus #{directory + "/" + item}`
    puts "processing " + item
    if (fileStatus.include? "up to date")
        puts item + " is up to date, starting to move file now."
        # cp command here. Something along this line: `cp #{directory + "/" + item + destination}`
        # rm command here. Probably you want to confirm that all copied files are correct by comparing md5 or something similar.
    else
        puts item + " is not up to date, moving on to next file."
    end
end

This is the full script, I ended up with:

# runs in Ruby 1.8.x (ftools)

require 'ftools'

directory = "path/to/dir"
destination = "location/to/move/to"

Dir.glob(directory+"/**/*") do |item|
    next if item == '.' or item == '..'
    fileStatus = `~/bin/dropbox.py filestatus #{item}`
    puts "processing " + item
    puts "filestatus: " + fileStatus
    if (fileStatus.include? "up to date")
        puts item.split('/',2)[1] + " is up to date, starting to move file now."
        `cp -r #{item + " " + destination + "/" + item.split('/',2)[1]}`

        # remove file in Dropbox folder, if current item is not a directory and 
        # copied file is identical.
        if (!File.directory?(item) && File.cmp(item, destination + "/" + item.split('/',2)[1]).to_s)
            puts "remove " + item
            `rm -rf #{item}`
        end
    else
        puts item + " is not up to date, moving to next file."
    end
end
cherrun
  • 1,862
  • 8
  • 30
  • 50