How to use Java to Index entire hard drive / filesystem?

Question

I want to use Java to get file changes on the entire hard drive. e.g. c:\ or /mnt/drives/hdd1

It is a requirement because many different computers are used with different file structure, which cannot be changed easily without having an impact on other software. But specific files and filetypes should be indexed. They can exist on drive c:\ d:\ e:\ and any subfolder.

Java WatchService doesn't do the job because you have to manually add every subdirectory. With over 10+k folders that is infeasible and SLOW.

I am searching for something like this in JAVA:

Difference between C# and Java implementation: If i ran the WatchService Java Code with Admin Privledges i can access c:\$Recycle.Bin but not c:\Documents and Settings. I get an Access Denied Exception. Can somebody tell me why? And as i mentioned WatchService is not the solution, because it takes a lot of time untill all subdirectories are crawled.. And registering every subfolder and maintaining a Map with pairs of WatchKey and java.nio2.Path is a very bad solution for 10,000+ folders.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace FileSystemWatcherTest
{
    class Program
    {
        static void Main(string[] args)
        {
            FileSystemWatcher fileSystemWatcher = new FileSystemWatcher();
            try
            {
                // Watch for changes on this path
                fileSystemWatcher.Path = "c:\\";

                // Watch for changes on all files
                fileSystemWatcher.Filter = "*.*";

                // Also watch for changes within sub directories
                fileSystemWatcher.IncludeSubdirectories = true;

                fileSystemWatcher.Changed += fileSystemWatcher_Changed;
                fileSystemWatcher.Created += fileSystemWatcher_Created;
                fileSystemWatcher.Deleted += fileSystemWatcher_Deleted;
                fileSystemWatcher.Renamed += fileSystemWatcher_Renamed;

                // Begin watching
                fileSystemWatcher.EnableRaisingEvents = true;

            }
            catch (Exception ex)
            {
                Console.WriteLine("An error occurred: " + ex.Message);
            }

            while (true)
            {
                System.Threading.Thread.Sleep(60 * 1000);
            }

        }

        static void fileSystemWatcher_Renamed(object sender, RenamedEventArgs e)
        {
            Console.WriteLine("Rename " + e.FullPath);
        }

        static void fileSystemWatcher_Deleted(object sender, FileSystemEventArgs e)
        {
            Console.WriteLine("Delete " + e.FullPath);
        }

        static void fileSystemWatcher_Created(object sender, FileSystemEventArgs e)
        {
            Console.WriteLine("Create " + e.FullPath);
        }

        static void fileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
        {
            Console.WriteLine("Change " + e.FullPath);
        }
    }
}

JAVA Implementation:

private void registerDirectoryWithSubfolders(final Path start) throws IOException {
    // register directory and sub-directories
    Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
        @Override
        public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
            throws IOException
        {
            System.out.println(dir);
            try
            {
                registerDirectory(dir);
            }
            catch(java.nio.file.AccessDeniedException ex)
            {
                System.err.println("Access Denied: " + dir);
            }
            catch(java.lang.Throwable ex)
            {
                System.err.println("Exception: " + dir);
            }

            return FileVisitResult.CONTINUE;
        }

        @Override
        public FileVisitResult visitFileFailed(Path file, IOException exc)
            throws IOException
        {
            System.err.println("Error And SKIP " + file);
            try {
                Thread.sleep(2000);
            } catch (InterruptedException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            return FileVisitResult.SKIP_SUBTREE;
        }
    });
}

Answer 1

To me this looks like a duplicate of Monitor subfolders with a Java watch service

You can watch subdirectories like this:

/**
 * Register the given directory, and all its sub-directories, with the WatchService.
 */
private void registerAll(final Path start) throws IOException {
    // register directory and sub-directories
    Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
        @Override
        public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
            throws IOException {
                dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
                return FileVisitResult.CONTINUE;
        }
    });
}