1

I want to monitor a directory, and when a file appears there open it, process it and then move it to another directory. The problem is how to check that the other program is done writing it. With Java 7, I can use a WatchService from FileSystem, but I can only check when the files are created. What I want is to know when all file handles are released.

My first thought was that I could obtain an exclusive lock, but it turned out that it was possible to kick out another application while it was actually updating the file.

What is the preferred way to do this in Java? Thanks!

Improve this question
2
  • 1
    The only thing you can do is look for a file which indicates it program has finished with that one.
    – Peter Lawrey
    Commented Dec 27, 2011 at 9:59
  • @PeterLawrey: Unfortunately, my program should be able to pick up files from any program that could generate them, and there's no way for me to make them create additional files.
    – Anders Sjöqvist
    Commented Feb 16, 2012 at 10:16

2 Answers 2

Reset to default
1

The Watcher APIs currently allow you to see events when a file system object is created, modified or deleted in a watched directory. They don't tell you about other inotify events (on Linux). In fact, I don't think there is a way to do this in pure Java.

I was looking for a way to do this myself a few weeks ago and I came across a mail thread that suggested that you could write a custom implementation of the FileSystem api that provided a file watcher that supported other file system events. I decided not to pursue it because I had an alternative solution ... based on knowledge of how the files I am watching are being produced.

In my case, the files are produced by instruments that save image files to a shared drive. The solution is to watch the stream of "modified" events for a newly created file. When it stops and no more have been forthcoming for a couple of seconds (the "settling time"), then the file can be processed.

If this solution proves to be unreliable, the fallback is to implement the watching and initial processing (taking a snapshot of the file) in C / C++ using the inotify calls directly. This will allow me to directly observe the file close event.

Improve this answer
1
  • Thanks! That makes sense. What I ended up doing, was to abandon the WatchService altogether and instead take a snapshot of the files in the directory at a given point in time, and then a new one a few seconds later. The unaltered files in the intersection of these snapshots were then considered safe for processing. If I'll receive bug reports about this later I'll consider rewriting it, but for now I'll just assume that it meets the requirements.
    – Anders Sjöqvist
    Commented Feb 16, 2012 at 10:12
1

The simplest way for a filebased interface is:

  1. The sender writes the files with a changed filename (e.g. "example.xml_")
  2. When the sender has finished writing the file, he renames it (e.g. "example.xml_" to "example.xml")
  3. The receiver scans only for "*.xml"
Improve this answer
1
  • 1
    1) This only works when you have control of whatever is creating the files, and can modify it (if necessary) to rename them. 2) Scanning involves polling and is inefficient. Use WatchService.
    – Stephen C
    Commented Jan 3, 2012 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.