FileWatching · helpdesk (published)

I'm struggling to understand how FileWatching.watch_folder works. If I set up a small "server" that is listening to a folder according to

function watchserver(path)
    @info "Starting server watching for changes in ", path
    while true
        file, event = watch_folder(path, -1) # timeout inf
        file === nothing && continue
        @info "Got ", event, file
        sleep(1)
        if event.changed || event.renamed
            try
                println("doing the work") # display_data(joinpath(path, file))
            catch e
                @error "Plotting failed, got error" e
            end
        end
        event.timedout && return
    end
end

path = "/home/fredrikb/Documents/"
# path = ARGS[1]
task = watchserver(path)

I expect to get a signle event when something happens. Instead, I seem to get repeated events causing the println statement to be executed indefinitely. I suspect my understanding of how watch_folder works might be wrong?

Fredrik Bagge Carlson (Jan 21 2021 at 18:01):

I expect watch_folder to be blocking, and return something when an event occurs?

Fredrik Bagge Carlson (Jan 21 2021 at 18:03):

clinton ~/scratch> julia6 data_verification_app.jl "/home/fredrikb/Documents/"
[ Info: ("Starting server watching for changes in ", "/home/fredrikb/Documents/")
[ Info: ("Got ", FileWatching.FileEvent(true, false, false), "2021-01-19 - HKM (another copy).zip")
2021-01-19 - HKM (another copy).zip
[ Info: ("Got ", FileWatching.FileEvent(false, true, false), "2021-01-19 - HKM (another copy).zip")
2021-01-19 - HKM (another copy).zip
[ Info: ("Got ", FileWatching.FileEvent(false, true, false), "2021-01-19 - HKM (another copy).zip")
2021-01-19 - HKM (another copy).zip

Fredrik Bagge Carlson (Jan 21 2021 at 18:11):

If I save all files that have generated an event in a set and only proceed if it's an event for a new file, I seem to get what I want.

Fredrik Ekre (Jan 21 2021 at 18:14):

Fredrik Ekre (Jan 21 2021 at 18:21):

For a new file it looks like you get two events, which makes sense when you think about it: one event for creating the file (.renamed) and one event for writing the content to the file (.changed).
Appending to an existing file gives one event (.changed) and deleting a file gives one event (.renamed).

Fredrik Bagge Carlson (Jan 21 2021 at 18:28):

That seems to agree with my experimentation. This was a bit trickier than I anticipated. It turns out that after the first handled file event that induces precompilation, subsequent file events are triggered too fast for the file to be written completely and reading it fails without some sleep before ^^

Fredrik Bagge Carlson (Jan 21 2021 at 18:29):

const handled = Set{String}()
function watchserver(path)
    @info "Starting server watching for changes in " path
    while true
        file, event = watch_folder(path, -1) # timeout inf
        (file === nothing || file ∈ handled) && continue
        if event.changed
            @info "Got " event file
            push!(handled, file)
            try
                fullpath = joinpath(path, file)
                # println(file)
                sleep(2)
                display_data(fullpath)
            catch e
                @error "Plotting failed, got error" e
            end
        end
        event.timedout && return
    end
end

# path = "/home/fredrikb/Documents/"
path = ARGS[1]
task = watchserver(path)

Stream: helpdesk (published)

Topic: FileWatching

Fredrik Bagge Carlson (Jan 21 2021 at 18:01):

Fredrik Bagge Carlson (Jan 21 2021 at 18:01):

Fredrik Bagge Carlson (Jan 21 2021 at 18:03):

Fredrik Bagge Carlson (Jan 21 2021 at 18:11):

Fredrik Ekre (Jan 21 2021 at 18:14):

Fredrik Ekre (Jan 21 2021 at 18:21):

Fredrik Bagge Carlson (Jan 21 2021 at 18:28):

Fredrik Bagge Carlson (Jan 21 2021 at 18:29):

Fredrik Bagge Carlson (Jan 21 2021 at 18:36):

Max Köhler (Jan 21 2021 at 19:37):

TheCedarPrince (Jan 22 2021 at 12:23):