Optimize file system enumeration pattern matching with specialized delegates

[Copilot]

Description

File system enumeration was re-checking pattern types on every entry to determine if fast paths apply (*literal → EndsWith). This moves pattern analysis to enumerable creation time and creates specialized delegates upfront.

Changes:

• Add GetPredicate<T> that analyzes patterns once and returns optimized FileSystemEnumerable<T>.FindPredicate delegates:
  • * → always-true predicate (with IsDirectory/IsFile check)
  • literal → Equals for exact filename matching (e.g., "log.txt" across directory hierarchies)
  • *literal → EndsWith (existing optimization, moved earlier)
  • literal* → StartsWith
  • *literal* → Contains
  • prefix*suffix → StartsWith + EndsWith with minimum length check
• Complex patterns fall back to full NFA-based matching
• Extract wildcard constants as SearchValues<char> (s_simpleWildcards, s_extendedWildcards) to internal static readonly fields in FileSystemName for shared use
• Combine IsDirectory/IsFile check with pattern matcher into a single delegate invocation (avoids double delegate call)
• Capture only expression in lambdas and compute span slices inline on each invocation to avoid string allocations and minimize capture overhead
• Switch over (useExtendedWildcards, entryType) tuple to return distinct delegates without capturing unnecessary variables
• Restructured pattern matching logic to use switch statement on (startsWithStar, endsWithStar) tuple for cleaner code organization

Customer Impact

Performance improvement for file enumeration with common glob patterns. Each file entry now calls a simple string operation instead of re-evaluating pattern structure and potentially running the full matching algorithm.

Regression

No. This is a performance optimization.

Testing

• All existing FileSystem tests pass
• Added unit test cases in FileSystemNameTests.cs for literal patterns, literal* (StartsWith), *literal* (Contains), and prefix*suffix patterns
• Added end-to-end integration tests in PatternTransformTests.cs that reference existing SimpleMatchData and Win32MatchData theory data from FileSystemNameTests, creating actual files and validating enumeration results for each pattern type through the full enumeration path including the optimized delegates

Risk

Low. The optimization preserves existing behavior by falling back to full pattern matching for any pattern not matching the fast paths. All existing tests pass unchanged.

Original prompt

When doing file system enumeration, every entry returned by the is is then checked to see whether it matches the caller-provided pattern. That's handled by a delegate that eventually calls into a Match Pattern routine. That routine has special-case, eg if the pattern is *literal it'll just do EndsWith. But that means that on every entry it's rechecking whether any of those fast paths apply. Instead, the entry points that create those delegates could create different delegates for each of the most important special-case, such that each call doesn't need to recheck the condition. It'll also be good to add additional special-case. For example, right now it special cases *literal to use EndsWith, but not literal* to use Starts With, and not *literal* to use Contains. Please optimize all of these things.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

[stephentoub]

@EgorBot -amd -intel -arm

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);

[HideColumns("Job", "Error", "StdDev", "Median", "RatioSD")]
[MemoryDiagnoser(false)]
public partial class Tests
{
    private string _directoryPath;

    [GlobalSetup]
    public void Setup()
    {
        _directoryPath = Path.GetTempFileName();
        File.Delete(_directoryPath);

        Directory.CreateDirectory(_directoryPath);
        for (int i = 0; i < 10_000; i++)
        {
            File.Create(Path.Combine(_directoryPath, $"file{i}.txt")).Dispose();
        }
    }

    [Benchmark]
    public int Sum1()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath)) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int Sum2()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "file*")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int Sum3()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "*.txt")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int Sum4()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "*12*")) sum += path.Length;
        return sum;
    }

    [GlobalCleanup]
    public void Cleanup()
    {
        Directory.Delete(_directoryPath, true);
    }
}

[Copilot]

@copilot, let's add the literal case where there are no wildcards, only literals
That can show in cases where eg a particular file (eg "log.txt") might show up in various levels of a directory hierar...

Added in 539b9a2. Literal patterns (no wildcards, no escape characters) now use a simple Equals call. This is useful for finding specific files like "log.txt" at various levels in a directory hierarchy.

[stephentoub]

@EgorBot -amd -intel -arm

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);

[HideColumns("Job", "Error", "StdDev", "Median", "RatioSD")]
[MemoryDiagnoser(false)]
public partial class Tests
{
    private string _directoryPath;

    [GlobalSetup]
    public void Setup()
    {
        _directoryPath = Path.GetTempFileName();
        File.Delete(_directoryPath);

        Directory.CreateDirectory(_directoryPath);
        for (int i = 0; i < 10_000; i++)
        {
            File.Create(Path.Combine(_directoryPath, $"file{i}.txt")).Dispose();
        }
    }

    [Benchmark]
    public int NoPattern()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath)) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int LiteralStar()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "file*")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int StarLiteral()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "*.txt")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int StarLiteralStar()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "*12*")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int LiteralStarLiteral()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "12*.txt")) sum += path.Length;
        return sum;
    }

    [GlobalCleanup]
    public void Cleanup()
    {
        Directory.Delete(_directoryPath, true);
    }
}

[stephentoub]

@copilot

[stephentoub]

@EgorBot -amd -intel -arm

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);

[HideColumns("Job", "Error", "StdDev", "Median", "RatioSD")]
[MemoryDiagnoser(false)]
public partial class Tests
{
    private string _directoryPath;

    [GlobalSetup]
    public void Setup()
    {
        _directoryPath = Path.GetTempFileName();
        File.Delete(_directoryPath);

        Directory.CreateDirectory(_directoryPath);
        for (int i = 0; i < 10_000; i++)
        {
            File.Create(Path.Combine(_directoryPath, $"file{i}.txt")).Dispose();
        }
    }

    [Benchmark]
    public int NoPattern()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath)) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int Literal()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "file123.txt")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int LiteralStar()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "file*")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int StarLiteral()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "*.txt")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int StarLiteralStar()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "*12*")) sum += path.Length;
        return sum;
    }

    [Benchmark]
    public int LiteralStarLiteral()
    {
        int sum = 0;
        foreach (string path in Directory.EnumerateFiles(_directoryPath, "12*.txt")) sum += path.Length;
        return sum;
    }

    [GlobalCleanup]
    public void Cleanup()
    {
        Directory.Delete(_directoryPath, true);
    }
}