121: Filesystem, Paths, URLs, OS APIs, File Watching, and Safe File Handling

Learning objectives

You will learn to:

use node:fs/promises;
understand files, directories, metadata, permissions, and file descriptors at a practical level;
construct portable paths with node:path;
convert between filesystem paths and file URLs;
distinguish module directory and current working directory;
stream large files instead of buffering them completely;
perform safe temporary/atomic-like file workflows;
understand file watching limitations;
avoid path traversal vulnerabilities;
handle user-uploaded filenames safely.

Promise filesystem API

js
import { readFile, writeFile } from 'node:fs/promises';

const text = await readFile('./data.txt', 'utf8');

await writeFile('./copy.txt', text, 'utf8');

Use Promise APIs for modern async application code.

Synchronous filesystem remains appropriate in selected startup/build/CLI contexts.

Encoding matters

Without encoding:

js
const data = await readFile('./data.txt');

console.log(Buffer.isBuffer(data)); // true

With:

js
await readFile('./data.txt', 'utf8');

you receive a string.

Do not decode arbitrary binary data as UTF-8.

Buffers are covered deeply in lesson 122.

File size

readFile() buffers entire file in memory.

For a 5 GB upload, this is unacceptable.

Use streams:

js
import { createReadStream } from 'node:fs';

const stream = createReadStream('./large.bin');

Later stream lesson explains backpressure.

File metadata

js
import { stat } from 'node:fs/promises';

const info = await stat('./data.txt');

console.log({
  size: info.size,
  file: info.isFile(),
  directory: info.isDirectory(),
  mtime: info.mtime,
});

Do not use metadata checked long before access as a security guarantee; filesystem state can change between check and use.

Directories

js
import { mkdir, readdir } from 'node:fs/promises';

await mkdir('./data/tasks', {
  recursive: true,
});

const entries = await readdir('./data/tasks', {
  withFileTypes: true,
});

for (const entry of entries) {
  console.log(entry.name, entry.isDirectory());
}

Delete/rename/copy

js
import {
  rename,
  rm,
  copyFile,
} from 'node:fs/promises';

Be explicit with recursive destructive operations.

Validate target paths before rm({ recursive: true }).

`node:path`

Portable join:

js
import path from 'node:path';

const file = path.join('data', 'tasks', '123.json');

Do not build filesystem paths by concatenating /:

js
'data/' + userInput

because separators/platform behavior and security are harder to reason about.

Resolve

js
const absolute = path.resolve('data', 'tasks');

Resolution is relative to:

js
process.cwd()

unless an absolute segment is supplied.

Module-relative:

js
const file = new URL('./data/config.json', import.meta.url);

Many Node filesystem APIs accept file URLs.

This can be an elegant ESM pattern.

File URLs

js
const url = new URL('./template.html', import.meta.url);

const html = await readFile(url, 'utf8');

When a library requires a path string:

js
import { fileURLToPath } from 'node:url';

const filename = fileURLToPath(url);

Do not manually strip file://.

URL encoding, Windows drive letters, and spaces make manual conversion incorrect.

Path normalization is not authorization

Suppose server serves files under:

text
/srv/uploads

User supplies:

text
../../etc/passwd

Danger:

js
const target = path.join('/srv/uploads', userPath);
return readFile(target);

You need to enforce containment.

Example concept:

js
const root = path.resolve('/srv/uploads');
const target = path.resolve(root, userPath);

const relative = path.relative(root, target);

if (
  relative.startsWith('..') ||
  path.isAbsolute(relative)
) {
  throw new Error('Path escapes upload root');
}

Also consider symlinks and threat model. For security-sensitive file serving, use carefully reviewed abstractions and platform controls.

User filenames

Do not trust:

text
../../report.pdf
C:\Windows\...
<script>.html
very-long-name
confusable Unicode

For stored uploads:

generate server-side storage key;
retain original name as metadata if needed;
validate length/type;
avoid using original name as path authority.

Example:

js
const storageName = crypto.randomUUID();

Store:

js
{
  originalName,
  storageName,
  mediaType,
  size
}

Extension is not content type

A file named:

text
image.jpg

may not contain JPEG data.

Server-side upload validation should consider:

size;
claimed MIME;
content signature/magic bytes where relevant;
malware/scanning policy;
safe serving headers.

Do not execute uploaded files.

Atomic replacement pattern

To reduce partially written configuration/data files:

text
write temporary file
fsync if durability requirement
rename temp → final

Rename within the same filesystem is commonly atomic at filesystem level, but exact durability semantics depend on OS/filesystem.

Simple pattern:

js
const temp = `${target}.${process.pid}.tmp`;

await writeFile(temp, JSON.stringify(data));

await rename(temp, target);

Production durability may require stronger fsync/directory handling.

For a database, do not reinvent transaction storage with JSON files.

Exclusive create

For some lock/create-once workflows:

js
await writeFile(path, data, {
  flag: 'wx',
});

fails if file exists.

Filesystem locking across processes is subtle. Use dedicated libraries/OS facilities where correctness matters.

File descriptors

Low-level:

js
const handle = await open(file, 'r');

try {
  ...
} finally {
  await handle.close();
}

Always close resources.

Higher-level APIs often manage descriptors for you.

Resource leaks can exhaust file descriptors.

File permissions

Unix-like modes use permission bits.

Example:

js
await writeFile(secretPath, secret, {
  mode: 0o600,
});

Platform semantics vary.

Application permissions should complement—not replace—OS/container access controls.

Temporary directory

js
import os from 'node:os';
import path from 'node:path';

const tempRoot = os.tmpdir();

Create unique temporary directories/files safely rather than predictable shared names.

Node filesystem APIs provide temporary directory helpers.

Clean up resources.

OS APIs

js
import os from 'node:os';

console.log({
  platform: os.platform(),
  arch: os.arch(),
  cpus: os.availableParallelism?.() ?? os.cpus().length,
  totalMemory: os.totalmem(),
  freeMemory: os.freemem(),
});

Do not use machine CPU count blindly to size database connection pools or worker concurrency. Container CPU quotas and workload constraints matter.

Watching files

Node supports file watching.

Modern development:

bash
node --watch src/server.js

for process restart during development.

fs.watch() can observe filesystem changes:

js
import { watch } from 'node:fs';

const watcher = watch('./config', (eventType, filename) => {
  console.log(eventType, filename);
});

File watching is platform-dependent:

duplicate events;
missing filename in cases;
rename semantics;
network filesystems;
editor atomic-save behavior.

Do not use fs.watch as guaranteed durable business event delivery.

Chokidar

For richer cross-platform development/watch behavior, Chokidar is a common package.

Again: filesystem watch is not a message queue.

Globbing

Modern Node and ecosystem packages can match file patterns.

Packages include glob, globby.

Use for:

build scripts;
tooling;
file discovery.

Do not pass untrusted glob patterns into privileged filesystem scans without constraining scope.

File server example

js
async function loadPublicAsset(relativePath) {
  const root = path.resolve('./public');
  const target = path.resolve(root, relativePath);
  const rel = path.relative(root, target);

  if (rel.startsWith('..') || path.isAbsolute(rel)) {
    throw new Error('Invalid path');
  }

  return readFile(target);
}

For production static hosting, a mature server/CDN is usually preferable to a hand-written implementation.

Failure clinic

`readFile` huge file

Memory spike.

Relative path assumes cwd

Works locally, fails under service manager.

Original upload filename used as storage path

Traversal/collision risk.

`existsSync` then write

Race condition:

text
check
another process writes
you write

Use atomic operation/flags.

File watcher used as reliable queue

Events can coalesce/drop/duplicate.

Exercises

Read/write JSON with fs/promises.
Build module-relative path using import.meta.url.
Compare cwd-relative behavior by launching from another directory.
Build a safe path-containment helper and test ../.
Implement temp-write + rename.
Store an upload with generated server filename.
Watch a directory and record editor-save event behavior.
Stream a 1 GB file without loading it into memory.

Mastery checklist

Explain:

buffered read versus stream;
cwd versus module location;
path versus file URL;
traversal;
upload filenames;
atomic replacement;
descriptors;
permissions;
watch limitations;
OS metadata.