A Practical Guide to FileSystemWatcher - Handling Missed and Duplicate Events

FileSystemWatcher is the first API that comes up when monitoring file changes in .NET on Windows. It conveniently delivers file and directory creations, changes, deletions, and renames as events - but if you treat Created or Changed as completion notifications, you will quite routinely get burned by missed events, duplicate notifications, and reading half-written files.

In this article, we organize how to use FileSystemWatcher and its pitfalls, assuming mainly file-based integration with .NET on Windows. For the underlying mutual-exclusion concepts, see also Mutual Exclusion Fundamentals for File-Based Integration - Best Practices for File Locks and Atomic Claims.

In reality, Created can fire while a file is still being copied, and Changed is by no means guaranteed to fire only once. When changes concentrate in a short window, the internal buffer can overflow and individual changes get dropped.

So the core of the design is this.

• Notifications are triggers
• The truth lives in a directory rescan
• Ownership comes from an atomic claim
• Idempotency catches whatever is left

In the body of the article, we walk through the traps of wiring FileSystemWatcher into file integration with this mindset.

The code in this article is published on GitHub as a complete buildable and runnable sample set (a library, a console demo that runs against a temporary directory, and unit tests that actually create and modify files to verify the events).

filesystemwatcher-safe-basics - komurasoft-blog-samples (GitHub)

Table of Contents

1. The Conclusion First (In One Line)
2. Common Misconception Patterns With FileSystemWatcher (Diagrams)
   • 2.1. Treating Created as a Completion Notification
   • 2.2. Trusting the Count and Order of Changed
   • 2.3. Losing Changes to Internal Buffer Overflow
3. Anti-Patterns
   • 3.1. Processing Directly Inside the Event Handler
   • 3.2. Trying to Reconstruct the True State From the Event Stream
   • 3.3. Treating “No More Changed” as Completion
   • 3.4. Believing a Bigger InternalBufferSize Solves It
   • 3.5. Logging Error and Ignoring It
4. Best Practices
   • 4.1. Fold Notifications Into “Rescan Requests”
   • 4.2. Make Completion Explicit on the Sender Side
   • 4.3. The Receiver Takes a Claim Atomically
   • 4.4. Full Rescan on Startup / Overflow / Reconnection
   • 4.5. Assume Idempotency
5. Pseudocode (Excerpts)
   • 5.1. The Typical Failure Pattern
   • 5.2. An Example in the Right Direction (Roughly Sketched)
6. A Rough Guide to Choosing
7. Conclusion
8. References

1. The Conclusion First (In One Line)

• FileSystemWatcher events are not completion notifications - they are hints that something changed
• Created / Changed / Renamed can duplicate, arrive in an order you did not expect, and get dropped on overflow
• Event handlers are more stable when they do no heavy work and only enqueue a rescan request
• Completion detection should be made explicit via temp -> close -> rename / replace or done files / manifests
• With multiple workers, you must take a claim atomically before reading
• Tuning InternalBufferSize is an aid. In the end, full rescans and idempotency are what work

In short: do not treat FileSystemWatcher as “a truthful history stream.” Things break far less if notifications remain nothing more than a signal of “time to go look.”

2. Common Misconception Patterns With FileSystemWatcher (Diagrams)

2.1. Treating Created as a Completion Notification

This is the most obvious landmine. During copies and transfers, Created fires the moment the file is created, and one or more Changed events may follow afterwards.

sequenceDiagram
    participant Sender as Sender
    participant Share as watched dir
    participant W as FileSystemWatcher
    participant Receiver as Receiver

    Sender->>Share: Creates orders.csv
    Share-->>W: Created
    W-->>Receiver: OnCreated
    Receiver->>Share: Opens and reads orders.csv
    Note over Receiver: Copy still in progress
    Sender->>Share: Writes the rest
    Share-->>W: Changed
    Share-->>W: Changed
    Note over Receiver: Missing rows / corrupt JSON / corrupt ZIP

Created may mean “the name became visible,” but it does not guarantee “it is now safe to read.” If you equate the two, you step on the same mine as section 2.1 of the previous article, just by a different route.

2.2. Trusting the Count and Order of Changed

Changed is not guaranteed to fire exactly once. Even ordinary operations like moving or saving can show up as multiple events. On top of that, you can also pick up the touches of antivirus software and indexers.

sequenceDiagram
    participant App as Saving app
    participant Dir as watched dir
    participant AV as AV / indexer
    participant W as FileSystemWatcher

    App->>Dir: Starts saving report.xlsx
    Dir-->>W: Created
    Dir-->>W: Changed
    App->>Dir: Rename from a temp file
    Dir-->>W: Renamed
    Dir-->>W: Changed
    AV->>Dir: Scan / attribute access
    Dir-->>W: Changed
    Note over W: Not guaranteed to be once-only or in this order

Expectations like “one Changed means done” or “nothing touches the file after Renamed” are quite precarious.

Additional notes:

• A file rename can produce a Changed event
• RenamedEventArgs.Name can be null when the OS cannot correlate the old/new names
• Hidden files are not exempt. “It’s a hidden temp name, so it won’t be seen” does not hold
• If the watched directory itself is renamed, that change is not reported

2.3. Losing Changes to Internal Buffer Overflow

FileSystemWatcher has an internal buffer. When changes concentrate in a short window, it overflows and individual notifications get dropped.

flowchart LR
    A[Many changes in a short window] --> B[Notifications pile up in the internal buffer]
    B --> C{Can processing keep up?}
    C -- Yes --> D[Process individual events in order]
    C -- No --> E[Overflow]
    E --> F[Error event]
    F --> G[Stop trusting the completeness of the per-event history]
    G --> H[Full rescan of the directory]

The important point here is that “an overflow means losing just one event” is not guaranteed. The completeness of the entire individual-event stream becomes suspect, so it is best to simply re-examine the whole picture.

3. Anti-Patterns

3.1. Processing Directly Inside the Event Handler

This puts too much weight - completion detection and ownership acquisition - on the events themselves.

watcher.Created += (_, e) =>
{
    using var stream = File.OpenRead(e.FullPath);
    Import(stream); // May still be mid-copy
};

watcher.Error += (_, e) =>
{
    Console.WriteLine(e.GetException()); // Just printing it
};

There are two problems.

• At the time of Created, the contents may be incomplete
• There is no recovery from failures or overflow

An event handler is at its best when it just raises a rescan request and returns immediately. If you start heavy I/O or DB updates here, you strangle yourself during bursts.

3.2. Trying to Reconstruct the True State From the Event Stream

The design of “add to a dictionary on Created, update on Changed, remove on Deleted, re-key on Renamed” looks clean at first glance. But once duplicates, splits, overflow, and external interference enter, the bookkeeping gradually stops adding up.

switch (e.ChangeType)
{
    case WatcherChangeTypes.Created:
        state[e.FullPath] = Pending;
        break;
    case WatcherChangeTypes.Changed:
        state[e.FullPath] = Modified;
        break;
    case WatcherChangeTypes.Deleted:
        state.Remove(e.FullPath);
        break;
}

Rather than struggling in this direction, it is stronger to re-check the actual files on disk each time. What matters in file integration is correctly finding what may be processed right now - not faithfully reconstructing the event history.

3.3. Treating “No More Changed” as Completion

This design smells just like the previous article’s “size stopped changing, so it’s done.” It looks convenient, but it determines completion by guessing.

if (lastChangedAt + TimeSpan.FromSeconds(10) < DateTime.UtcNow)
{
    return Ready;
}

Cases where this fails include:

• A large file copy pauses midway
• The sending app saves in multiple stages
• Notifications appear delayed over a network share
• An external process rewrites attributes or timestamps afterwards

Completion is more stable when stated explicitly rather than guessed.

3.4. Believing a Bigger InternalBufferSize Solves It

Tuning InternalBufferSize matters, but it is not the heart of the design.

• The default is 8192 bytes
• It cannot go below 4096 bytes, and cannot exceed 64 KB
• The buffer uses non-paged memory, so increasing it is not as casual as it sounds

In other words, even at 64 KB, a notification burst beyond it ends the story. And it does not solve the “is this a completion notification?” problem by even one millimeter.

Before enlarging the buffer, there are things to do first.

• Narrow the watch scope with Filter / Filters
• Keep NotifyFilter to the necessary minimum
• Do not set IncludeSubdirectories to true carelessly
• Keep the event handlers lightweight
• Add full rescans and idempotency

3.5. Logging Error and Ignoring It

Error is not the kind of notification you can “see occasionally and shrug off.” Buffer overflows and failures to continue watching surface here.

watcher.Error += (_, e) =>
{
    _logger.LogError(e.GetException(), "watcher error");
    // Ending here means noticing the loss but never recovering
};

At a minimum, you want to go this far.

• Request a full rescan
• If continued watching is in doubt, consider recreating the watcher
• Make reprocessing idempotent, on the assumption that events were lost

4. Best Practices

4.1. Fold Notifications Into “Rescan Requests”

Wiring Created / Changed / Deleted / Renamed / Error each directly into separate business logic ruins clarity. First fold them all into one kind of signal: “go look.”

flowchart LR
    A[Created / Changed / Deleted / Renamed] --> Q[scan request]
    B[Error / overflow] --> Q
    C[startup] --> Q
    Q --> D[Rescan the directory]
    D --> E[Enumerate ready candidates]
    E --> F[Attempt a claim]

Implementation points:

• In the event handler, do little more than set dirty = true and raise a signal
• Concentrate scanning in a single worker
• During bursts, coalesce for around 100-300 ms, then scan once
• If more notifications arrive during a scan, scan once more afterwards

This way, whether five events arrive or fifty, the final action is unified: “look at the actual files and find what is ready.”

4.2. Make Completion Explicit on the Sender Side

If you also control the sender, fixing the publishing protocol beats straining over completion detection on the FileSystemWatcher side.

The proven route, once again, is:

• Write the full content under a temp name
• close it
• rename / replace on the same file system
• If needed, place a done file / manifest last

flowchart TD
    A[Write the full content to data.tmp] --> B[flush / close]
    B --> C[rename / replace to data.csv]
    C --> D[Place data.done / manifest.json]
    D --> E[Receiver watches only final names or done files]

Same as the previous article, but this is where the real payoff is. The right way to see FileSystemWatcher is not as a tool that invents completion, but as a tool that notices explicitly declared completion sooner.

4.3. The Receiver Takes a Claim Atomically

Even when a rescan finds a ready candidate, going straight in to read lets multiple workers grab it simultaneously. So take a claim atomically before processing.

sequenceDiagram
    participant Scan as scanner
    participant IN as incoming
    participant P1 as processing/worker1
    participant P2 as processing/worker2

    Scan->>IN: Finds order-123
    Scan->>P1: rename order-123
    Scan->>P2: rename order-123
    Note over P1,P2: Only the one that succeeds first holds ownership

As mentioned in the previous article, the incoming -> processing/<worker>/ rename is the clearest. Bundling the payload + manifest + auxiliary files into one directory is especially convenient, since you can then claim per bundle.

incoming/
  order-123/
    payload.csv
    manifest.json

With this, a single rename of the bundle directory takes ownership.

4.4. Full Rescan on Startup / Overflow / Reconnection

This is quite important.

• Files placed before the app started are not picked up by events
• Once an overflow occurs, the individual event stream becomes hard to trust
• With network shares and transient disconnections in play, it is safer to assume “something in between” was missed

So at least at these moments, a full rescan should be performed.

• At startup
• On receiving Error
• Right after recreating the watcher
• Periodically, as insurance, at a fixed interval

The philosophy here: “the watcher is a hint about deltas; the rescan is the recovery of consistency.”

4.5. Assume Idempotency

With FileSystemWatcher, you will end up examining the same target multiple times. That is not a bug - it is more stable to accept it as part of the design.

Concretely, it goes like this.

• Put an IdempotencyKey in the manifest
• If already processed, do not re-execute the side effects
• Make archived / DB-recorded / sent statuses verifiable
• Ensure that a full rescan amounts to nothing more than “safely looking at the same things again”

Trying to build exactly-once out of events alone gets painful fast. Accepting at-least-once and closing the loop with idempotency is the stronger position in practice.

5. Pseudocode (Excerpts)

5.1. The Typical Failure Pattern

using var watcher = new FileSystemWatcher(incomingDir)
{
    Filter = "*.csv",
    IncludeSubdirectories = false,
    EnableRaisingEvents = true,
    InternalBufferSize = 64 * 1024
};

watcher.Created += (_, e) =>
{
    // Assumes Created = completion notification
    ProcessFile(e.FullPath);
};

watcher.Changed += (_, e) =>
{
    // It keeps firing, so just process again
    ProcessFile(e.FullPath);
};

watcher.Error += (_, e) =>
{
    Console.WriteLine(e.GetException());
    // No recovery
};

There are four problems.

• Created / Changed are wired directly into business processing
• There is no completion detection
• No full rescan on overflow
• No mechanism to stop processing the same file repeatedly

5.2. An Example in the Right Direction (Roughly Sketched)

private readonly SemaphoreSlim _scanSignal = new(0, int.MaxValue);
private int _scanRequested = 0;
private int _fullRescanRequested = 0;

void OnAnyChange(object? sender, FileSystemEventArgs e)
{
    RequestScan(full: false);
}

void OnRenamed(object? sender, RenamedEventArgs e)
{
    RequestScan(full: false);
}

void OnError(object? sender, ErrorEventArgs e)
{
    Log(e.GetException());
    RequestScan(full: true);
}

void RequestScan(bool full)
{
    if (full)
    {
        Interlocked.Exchange(ref _fullRescanRequested, 1);
    }

    if (Interlocked.Exchange(ref _scanRequested, 1) == 0)
    {
        _scanSignal.Release();
    }
}

async Task ScannerLoopAsync(CancellationToken cancellationToken)
{
    RequestScan(full: true); // startup scan

    while (!cancellationToken.IsCancellationRequested)
    {
        await _scanSignal.WaitAsync(cancellationToken);

        // Coalesce notification bursts a little
        await Task.Delay(TimeSpan.FromMilliseconds(200), cancellationToken);

        Interlocked.Exchange(ref _scanRequested, 0);
        bool full = Interlocked.Exchange(ref _fullRescanRequested, 0) == 1;

        foreach (var bundle in EnumerateReadyBundles(incomingDir, full))
        {
            var claimedPath = Path.Combine(processingDir, bundle.Name);

            if (!TryClaimByRename(bundle.Path, claimedPath))
            {
                continue; // Another worker claimed it first
            }

            var manifest = ReadManifest(Path.Combine(claimedPath, "manifest.json"));

            if (AlreadyProcessed(manifest.IdempotencyKey))
            {
                MoveToArchive(claimedPath, archiveDir);
                continue;
            }

            ProcessBundle(claimedPath);
            RecordProcessed(manifest.IdempotencyKey);
            MoveToArchive(claimedPath, archiveDir);
        }

        if (Volatile.Read(ref _scanRequested) == 1)
        {
            _scanSignal.Release(); // Do not drop notifications that arrived mid-scan
        }
    }
}

What matters in this example is the flow, not the fine API details.

• Fold notifications into scan requests
• Find what is ready by scanning
• Take a claim
• Check idempotency
• Process, record, and move to the archive

The FileSystemWatcher events are nothing more than a trigger here.

6. A Rough Guide to Choosing

• Single receiving worker / you can also fix the sender Start with temp -> close -> rename and a startup scan. That alone gets you quite far.

• Multiple receiving workers Add the incoming -> processing claim rename on top of the above.

• High-frequency, notification-heavy Narrow Filter / NotifyFilter / IncludeSubdirectories and minimize the event handlers. Tuning InternalBufferSize comes after that.

• Overflows hurt / missed events are unacceptable Build on full rescans, and if that is still not enough, do not bet on FileSystemWatcher alone. If you are Windows-only, the USN change journal is also an option.

• You cannot control how the other system writes Rather than papering over completion conditions with guesses, first consider whether the publishing protocol can be negotiated. If not, lower the guarantee level and lean into an idempotent receiving design.

The last two items are fairly important withdrawal criteria. FileSystemWatcher is useful, but it is not an all-powerful truth detector.

7. Conclusion

FileSystemWatcher is no substitute for completion notifications. The truth is not in the event stream but in what is visible on disk right now. Make completion explicit via temp -> close -> rename / replace or done files / manifests, and decide ownership by taking a claim atomically. That is where the heart of the design lives.

Processing immediately on Created, trusting the count or order of Changed, treating “no more Changed” as completion, taking comfort in InternalBufferSize alone, seeing Error and never recovering - all designs to avoid. Instead, fold notifications into rescan requests, perform full rescans on startup / overflow / reconnection, take ownership through claim renames, and absorb duplicates and re-scans with idempotency.

In other words, the trick with FileSystemWatcher is to never equate “having received an event” with “being allowed to process.” Just separating those two greatly reduces the kind of monitoring code that breaks only once in a while.

8. References

• Complete sample code for this article (library, demo, unit tests) https://github.com/gomurin0428/komurasoft-blog-samples/tree/main/filesystemwatcher-safe-basics
• Related article: Mutual Exclusion Fundamentals for File-Based Integration - Best Practices for File Locks and Atomic Claims
• FileSystemWatcher Class (System.IO)
• System.IO.FileSystemWatcher class - .NET
• FileSystemWatcher.InternalBufferSize Property (System.IO)
• FileSystemWatcher.Error Event (System.IO)
• FileSystemWatcher.Created Event (System.IO)
• FileSystemWatcher.Changed Event (System.IO)
• FileSystemWatcher.Renamed Event (System.IO)
• Change Journals - Win32 apps

Related Articles

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

Related Topics

These topic pages place the article in a broader service and decision context.

Where This Topic Connects

This article connects naturally to the following service pages.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.