Skip to content

Commit fd43e85

Browse files
committed
fixup! address review
1 parent 721c2e2 commit fd43e85

14 files changed

Lines changed: 983 additions & 194 deletions

doc/api/fs.md

Lines changed: 118 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1450,7 +1450,7 @@ changes:
14501450
* `options` {Object}
14511451
* `cwd` {string|URL} current working directory. **Default:** `process.cwd()`
14521452
* `exclude` {Function|string\[]} Function to filter out files/directories or a
1453-
list of glob patterns to be excluded. If a function is provided, return
1453+
list of [glob patterns][] to be excluded. If a function is provided, return
14541454
`true` to exclude the item, `false` to include it. **Default:** `undefined`.
14551455
If a string array is provided, each string should be a glob pattern that
14561456
specifies paths to exclude. Note: Negation patterns (e.g., '!foo.js') are
@@ -1462,6 +1462,8 @@ changes:
14621462
* Returns: {AsyncIterator} An AsyncIterator that yields the paths of files
14631463
that match the pattern.
14641464
1465+
See [Glob patterns][] for the syntax `pattern` accepts.
1466+
14651467
When `followSymlinks` is enabled, detected symbolic link cycles are not
14661468
traversed recursively.
14671469
@@ -2227,9 +2229,10 @@ added:
22272229
queued than `maxQueue` allows. `'ignore'` means overflow events are dropped and a
22282230
warning is emitted, while `'throw'` means to throw an exception. **Default:** `'ignore'`.
22292231
* `ignore` {string|RegExp|Function|Array} Pattern(s) to ignore. Strings are
2230-
glob patterns, RegExp patterns are tested against
2231-
the filename, and functions receive the filename and return `true` to
2232-
ignore. **Default:** `undefined`.
2232+
[glob patterns][] that, when they contain no `/`, are matched against the
2233+
file's basename; RegExp patterns are tested against the filename, and
2234+
functions receive the filename and return `true` to ignore.
2235+
**Default:** `undefined`.
22332236
* Returns: {AsyncIterator} of objects with the properties:
22342237
* `eventType` {string} The type of change
22352238
* `filename` {string|Buffer|null} The name of the file changed.
@@ -3641,7 +3644,7 @@ changes:
36413644
* `options` {Object}
36423645
* `cwd` {string|URL} current working directory. **Default:** `process.cwd()`
36433646
* `exclude` {Function|string\[]} Function to filter out files/directories or a
3644-
list of glob patterns to be excluded. If a function is provided, return
3647+
list of [glob patterns][] to be excluded. If a function is provided, return
36453648
`true` to exclude the item, `false` to include it. **Default:** `undefined`.
36463649
* `followSymlinks` {boolean} When `true`, symbolic links to directories are
36473650
followed while expanding `**` patterns. **Default:** `false`.
@@ -3653,6 +3656,8 @@ changes:
36533656
36543657
* Retrieves the files matching the specified pattern.
36553658
3659+
See [Glob patterns][] for the syntax `pattern` accepts.
3660+
36563661
When `followSymlinks` is enabled, detected symbolic link cycles are not
36573662
traversed recursively.
36583663
@@ -5300,9 +5305,10 @@ changes:
53005305
* `throwIfNoEntry` {boolean} Indicates whether an exception should be thrown when the
53015306
path does not exist. **Default:** `true`.
53025307
* `ignore` {string|RegExp|Function|Array} Pattern(s) to ignore. Strings are
5303-
glob patterns, RegExp patterns are tested against
5304-
the filename, and functions receive the filename and return `true` to
5305-
ignore. **Default:** `undefined`.
5308+
[glob patterns][] that, when they contain no `/`, are matched against the
5309+
file's basename; RegExp patterns are tested against the filename, and
5310+
functions receive the filename and return `true` to ignore.
5311+
**Default:** `undefined`.
53065312
* `listener` {Function|undefined} **Default:** `undefined`
53075313
* `eventType` {string}
53085314
* `filename` {string|Buffer|null}
@@ -6278,14 +6284,16 @@ changes:
62786284
* `options` {Object}
62796285
* `cwd` {string|URL} current working directory. **Default:** `process.cwd()`
62806286
* `exclude` {Function|string\[]} Function to filter out files/directories or a
6281-
list of glob patterns to be excluded. If a function is provided, return
6287+
list of [glob patterns][] to be excluded. If a function is provided, return
62826288
`true` to exclude the item, `false` to include it. **Default:** `undefined`.
62836289
* `followSymlinks` {boolean} When `true`, symbolic links to directories are
62846290
followed while expanding `**` patterns. **Default:** `false`.
62856291
* `withFileTypes` {boolean} `true` if the glob should return paths as Dirents,
62866292
`false` otherwise. **Default:** `false`.
62876293
* Returns: {string\[]} paths of files that match the pattern.
62886294
6295+
See [Glob patterns][] for the syntax `pattern` accepts.
6296+
62896297
When `followSymlinks` is enabled, detected symbolic link cycles are not
62906298
traversed recursively.
62916299
@@ -9201,6 +9209,101 @@ example `fs.readdirSync('C:\\')` can potentially return a different result than
92019209
`fs.readdirSync('C:')`. For more information, see
92029210
[this MSDN page][MSDN-Rel-Path].
92039211
9212+
### Glob patterns
9213+
9214+
[`fs.glob()`][], [`fs.globSync()`][], [`fsPromises.glob()`][] and
9215+
[`path.matchesGlob()`][] take glob patterns, as do the string forms of the
9216+
`exclude` and `ignore` options. The syntax follows the pattern matching of
9217+
`bash`, with brace expansion and the extended `extglob` operators.
9218+
9219+
The glob implementation Node.js uses was adopted from [`minimatch`][],
9220+
so as a general rule of thumb, all minimatch-supported glob extensions
9221+
work with `fs`. For simplicity, such extensions have been documented below:
9222+
9223+
#### Path separators
9224+
9225+
A pattern is always split on `/`, on every platform. A backslash in a pattern
9226+
is treated as a path separator as well, and never as an escape character, so
9227+
patterns built with `path.join()` on Windows still work. Repeated separators
9228+
are collapsed, so `a//b` and `a/b` are the same pattern.
9229+
9230+
#### Wildcards
9231+
9232+
Wildcards match within a single path segment, and never match a `/`:
9233+
9234+
| Pattern | Matches |
9235+
| ------------------ | ------------------------------------------ |
9236+
| `*` | Any run of characters, including none |
9237+
| `?` | Exactly one character |
9238+
| `[abc]` | Any one of the characters in the set |
9239+
| `[a-z]` | Any one character in the range |
9240+
| `[!abc]`, `[^abc]` | Any one character not in the set |
9241+
| `[[:alpha:]]` | Any one character in the named POSIX class |
9242+
9243+
```js
9244+
path.matchesGlob('src/index.js', 'src/*.js'); // true
9245+
path.matchesGlob('src/lib/index.js', 'src/*.js'); // false
9246+
path.matchesGlob('file1.txt', 'file[0-9].txt'); // true
9247+
path.matchesGlob('é', '[[:alpha:]]'); // true
9248+
```
9249+
9250+
#### Globstar
9251+
9252+
A `**` that makes up a whole path segment matches zero or more segments, so
9253+
`a/**/b` matches both `a/b` and `a/x/y/b`.
9254+
9255+
```js
9256+
path.matchesGlob('src/a/b/index.js', 'src/**/*.js'); // true
9257+
path.matchesGlob('src/index.js', 'src/**/*.js'); // true
9258+
```
9259+
9260+
#### Extended globs
9261+
9262+
Each of these takes a `|`-separated list of alternatives, and they may nest:
9263+
9264+
| Pattern | Matches |
9265+
| --------- | ------------------------------- |
9266+
| `?(a\|b)` | Zero or one of the alternatives |
9267+
| `*(a\|b)` | Zero or more of them |
9268+
| `+(a\|b)` | One or more of them |
9269+
| `@(a\|b)` | Exactly one of them |
9270+
| `!(a\|b)` | Anything except them |
9271+
9272+
```js
9273+
path.matchesGlob('index.ts', '*.@(js|ts)'); // true
9274+
path.matchesGlob('index.css', '!(*.js)'); // true
9275+
```
9276+
9277+
#### Brace expansion
9278+
9279+
Braces expand to alternatives before anything else in the pattern is
9280+
interpreted. `{a,b}` is a list, `{1..9}` and `{a..z}` are sequences, and a
9281+
sequence may take a step (`{1..9..3}`) and keep zero padding (`{01..12}`).
9282+
Braces nest. A brace group containing neither a comma nor a sequence is
9283+
literal.
9284+
9285+
```js
9286+
path.matchesGlob('src/index.ts', 'src/*.{js,ts}'); // true
9287+
path.matchesGlob('page3.html', 'page{1..5}.html'); // true
9288+
path.matchesGlob('a{b}c', 'a{b}c'); // true
9289+
```
9290+
9291+
#### Dot files
9292+
9293+
A path segment beginning with a `.` is matched only by a pattern segment
9294+
beginning with a literal `.`, so `*` and `**` do not match dot files or
9295+
directories.
9296+
9297+
```js
9298+
path.matchesGlob('.env', '*'); // false
9299+
path.matchesGlob('.env', '.*'); // true
9300+
```
9301+
9302+
#### Case sensitivity
9303+
9304+
Matching is case-sensitive, except on Windows and macOS, where the file system
9305+
is not case-sensitive itself.
9306+
92049307
### File descriptors
92059308

92069309
On POSIX systems, for every process, the kernel maintains a table of currently
@@ -9372,6 +9475,7 @@ the file contents.
93729475
[FS constants]: #fs-constants
93739476
[File access constants]: #file-access-constants
93749477
[File modes]: #file-modes
9478+
[Glob patterns]: #glob-patterns
93759479
[MDN-Date]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date
93769480
[MDN-Number]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Data_structures#number_type
93779481
[MSDN-Rel-Path]: https://docs.microsoft.com/en-us/windows/desktop/FileIO/naming-a-file#fully-qualified-vs-relative-paths
@@ -9401,6 +9505,8 @@ the file contents.
94019505
[`fs.fstat()`]: #fsfstatfd-options-callback
94029506
[`fs.ftruncate()`]: #fsftruncatefd-len-callback
94039507
[`fs.futimes()`]: #fsfutimesfd-atime-mtime-callback
9508+
[`fs.glob()`]: #fsglobpattern-options-callback
9509+
[`fs.globSync()`]: #fsglobsyncpattern-options
94049510
[`fs.lstat()`]: #fslstatpath-options-callback
94059511
[`fs.lutimes()`]: #fslutimespath-atime-mtime-callback
94069512
[`fs.mkdir()`]: #fsmkdirpath-options-callback
@@ -9429,6 +9535,7 @@ the file contents.
94299535
[`fs.writev()`]: #fswritevfd-buffers-position-callback
94309536
[`fsPromises.access()`]: #fspromisesaccesspath-mode
94319537
[`fsPromises.copyFile()`]: #fspromisescopyfilesrc-dest-mode
9538+
[`fsPromises.glob()`]: #fspromisesglobpattern-options
94329539
[`fsPromises.mkdtemp()`]: #fspromisesmkdtempprefix-options
94339540
[`fsPromises.open()`]: #fspromisesopenpath-flags-mode
94349541
[`fsPromises.opendir()`]: #fspromisesopendirpath-options
@@ -9437,7 +9544,9 @@ the file contents.
94379544
[`fsPromises.utimes()`]: #fspromisesutimespath-atime-mtime
94389545
[`inotify(7)`]: https://man7.org/linux/man-pages/man7/inotify.7.html
94399546
[`kqueue(2)`]: https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2
9547+
[`minimatch`]: https://github.com/isaacs/minimatch
94409548
[`node:stream/iter`]: stream_iter.md
9549+
[`path.matchesGlob()`]: path.md#pathmatchesglobpath-pattern
94419550
[`statfs.bsize`]: #statfsbsize
94429551
[`stream.getDefaultHighWaterMark()`]: stream.md#streamgetdefaulthighwatermarkobjectmode
94439552
[`stream/iter pipeTo()`]: stream_iter.md#pipetosource-transforms-writer-options

doc/api/path.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -302,6 +302,7 @@ changes:
302302
* Returns: {boolean} Whether or not the `path` matched the `pattern`.
303303

304304
The `path.matchesGlob()` method determines if `path` matches the `pattern`.
305+
See [Glob patterns][] for the syntax `pattern` accepts.
305306

306307
For example:
307308

@@ -657,6 +658,7 @@ of the `path` methods.
657658

658659
The API is accessible via `require('node:path').win32` or `require('node:path/win32')`.
659660

661+
[Glob patterns]: fs.md#glob-patterns
660662
[MSDN-Rel-Path]: https://docs.microsoft.com/en-us/windows/desktop/FileIO/naming-a-file#fully-qualified-vs-relative-paths
661663
[`TypeError`]: errors.md#class-typeerror
662664
[`path.parse()`]: #pathparsepath

src/glob/glob_ast.h

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010

1111
namespace node::glob {
1212

13+
// UTF-16, because that is how patterns arrive from V8
1314
using PatternString = std::u16string;
1415
using PatternView = std::u16string_view;
1516

@@ -44,15 +45,29 @@ bool IsWindowsDrive(std::basic_string_view<Char> s) {
4445
return s.size() == 2 && StartsWithWindowsDrive(s);
4546
}
4647

48+
// Caps on what one pattern may cost, inherited from minimatch
4749
inline constexpr size_t kMaxPatternLength = 64 * 1024;
50+
// Nesting depth for extglobs like `*(a|@(b|c))`
4851
inline constexpr int kMaxExtglobRecursion = 2;
52+
// Non-adjacent `**` parts one match may recurse through
4953
inline constexpr int kMaxGlobstarRecursion = 200;
54+
// How far, in number of braces, braces may expand
5055
inline constexpr size_t kBraceExpansionMax = 100'000;
56+
// How far, in length, braces may expand
5157
inline constexpr size_t kBraceExpansionMaxLength = 4'000'000;
58+
// Braces and extglobs are parsed by recursing once per level of nesting, so
59+
// nesting deep enough to exhaust the C stack has to be refused up front:
60+
// `@(` and `{,` cost two or three code units per level, which leaves room for
61+
// thousands of levels under kMaxPatternLength. Minimatch has no such cap and
62+
// simply throws RangeError once V8's stack runs out. Real patterns nest a
63+
// handful of levels, and 256 keeps the parser under ~150 KB of stack even on
64+
// the smallest worker thread stacks.
65+
inline constexpr int kMaxNestingDepth = 256;
5266

5367
enum class CompileError {
5468
kNone,
5569
kPatternTooLong,
70+
kPatternTooDeep,
5671
kInvalidRegExp,
5772
};
5873

src/glob/glob_matcher.cc

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
#include <algorithm>
44
#include <bit>
5+
#include <limits>
56
#include <ranges>
67
#include <span>
78

@@ -13,18 +14,25 @@ namespace node::glob {
1314

1415
namespace {
1516

16-
// A set of match positions within one segment.
17+
// The set of positions in the subject a segment's program can currently be
18+
// at. Carrying every position at once is what makes matching linear in
19+
// (positions x nodes) instead of backtracking, so a pattern like
20+
// `*(a|aa)*(a|aa)b` costs the same as any other.
1721
class PosSet {
1822
public:
1923
void Init(size_t positions) {
20-
nwords_ = (positions + 63) / 64;
24+
nwords_ = (positions + kBitsPerWord - 1) / kBitsPerWord;
2125
if (nwords_ > kInlineWords)
2226
heap_.assign(nwords_, 0);
2327
else
2428
std::fill_n(inline_, nwords_, 0);
2529
}
26-
void Set(size_t i) { words()[i >> 6] |= uint64_t{1} << (i & 63); }
27-
bool Test(size_t i) const { return (words()[i >> 6] >> (i & 63)) & 1; }
30+
void Set(size_t i) {
31+
words()[i >> kWordShift] |= uint64_t{1} << (i & kBitInWordMask);
32+
}
33+
bool Test(size_t i) const {
34+
return (words()[i >> kWordShift] >> (i & kBitInWordMask)) & 1;
35+
}
2836
bool Empty() const {
2937
return std::ranges::all_of(words(), [](uint64_t v) { return v == 0; });
3038
}
@@ -34,7 +42,7 @@ class PosSet {
3442
for (size_t w = 0; w < nwords_; w++) {
3543
uint64_t word = ws[w];
3644
while (word != 0) {
37-
f(w * 64 + std::countr_zero(word));
45+
f(w * kBitsPerWord + std::countr_zero(word));
3846
word &= word - 1;
3947
}
4048
}
@@ -46,7 +54,12 @@ class PosSet {
4654
}
4755

4856
private:
49-
static constexpr size_t kInlineWords = 4; // 256 positions
57+
static constexpr size_t kBitsPerWord = std::numeric_limits<uint64_t>::digits;
58+
static constexpr size_t kWordShift = std::countr_zero(kBitsPerWord);
59+
static constexpr size_t kBitInWordMask = kBitsPerWord - 1;
60+
// Four words hold 256 positions, one more than the longest name the
61+
// common filesystems allow, so only a synthetic subject reaches the heap.
62+
static constexpr size_t kInlineWords = 4;
5063

5164
std::span<uint64_t> words() {
5265
return {nwords_ <= kInlineWords ? inline_ : heap_.data(), nwords_};
@@ -306,7 +319,7 @@ bool EndsWith(std::basic_string_view<Char> s, PatternView suffix) {
306319
template <typename Char>
307320
bool EndsWithLowered(std::basic_string_view<Char> f, PatternView lowered_ext) {
308321
const bool ascii = std::ranges::all_of(
309-
f, [](Char c) { return static_cast<uint32_t>(c) < 0x80; });
322+
f, [](Char c) { return static_cast<uint32_t>(c) <= kMaxAsciiCodePoint; });
310323
if (ascii) {
311324
return f.size() >= lowered_ext.size() &&
312325
std::ranges::equal(lowered_ext,
@@ -446,6 +459,8 @@ class PathMatcher {
446459

447460
private:
448461
using RowParts = std::vector<PartMatcher>;
462+
// Path depth held without allocating. Deeper paths still work; they just
463+
// grow the buffer, and they are rare enough not to matter.
449464
static constexpr size_t kInlineParts = 32;
450465

451466
// Minimatch#slashSplit

0 commit comments

Comments
 (0)