[RFC] Move from global `env` to session-based configuration

[nico-martin]

Summary

Currently, Transformers.js uses a global env object for configuration. I propose moving to a session-based approach where configuration is passed directly to pipeline instances, while maintaining backward compatibility with the existing env object as defaults.

Current Behavior

import { env, pipeline } from "@huggingface/transformers";

env.remoteHost = "https://myhost.com";

const pipe = await pipeline('sentiment-analysis', 'Xenova/bert-base-multilingual-uncased-sentiment');
// All pipelines now use myhost.com

All pipelines share the same global configuration through the env object.

Proposed Behavior

import { pipeline } from "@huggingface/transformers";

const pipe = await pipeline('sentiment-analysis', 'Xenova/bert-base-multilingual-uncased-sentiment', {
  remoteHost: "https://myhost.com",
});
// Only this pipeline uses myhost.com

Configuration is passed per-pipeline, allowing different instances to use different settings.

Motivation

1. Client-side context: The env concept feels server-oriented, where configuration is typically set once at startup. On the client, where you have long-running processes and lots of user interaction, a session-based approach makes more sense.

2. Better encapsulation: Different pipeline instances might need different configurations (e.g., loading some models from a local cache, others from a CDN, others from a private host).

3. Industry standard: Other similar libraries follow this pattern:

   • MediaPipe.js
   • TensorFlow.js
   • WebLLM

   They all use session/instance-based configuration rather than global state.

4. Semantic clarity: An "environment" (env) should represent static values based on the environment an app runs in (e.g., IS_BROWSER_ENV, IS_FS_AVAILABLE). Most of the current env properties are actually runtime configuration options, not environment detection.

Backward Compatibility

The existing env object would serve as the default configuration:

import { env, pipeline } from "@huggingface/transformers";

env.remoteHost = "https://myhost.com";

// Uses env.remoteHost as default
const pipe1 = await pipeline("text-generation", "gpt2");

// Overrides the default for this instance
const pipe2 = await pipeline("text-generation", "gpt2", {
  remoteHost: "https://mysecondhost.com",
});

This would maintain full backward compatibility while enabling the new pattern.

Implementation Considerations

I recognize this is a significant change. However, with v4 on the horizon, it might be a good opportunity to introduce this improvement. The migration path would be smooth since:

1. Existing code continues to work unchanged
2. Users can gradually adopt the new pattern
3. Clear migration guide can be provided

Questions

• Would this fit into the v4 roadmap?
• Are there use cases where the global env pattern is preferable?
• Should we consider deprecation warnings for certain env properties in v4, with removal in v5?

[xenova]

Are there use cases where the global env pattern is preferable?

Originally, the env property was created to help provide a useful way to access the underlying ORT env variable (now under env.backends.onnx). For ORT, there are certain global session variables, like whether to use a proxy, wasm paths, etc. And these do indeed feel necessary to remain as global/singleton values.

Another consideration is alignment with the python transformers library. A global env is already a bit non-standard, and I fear that adding these to per-session options would be even more complicated. Using the example you use above, I don't think there are enough users who would need multiple remote host locations on a per-session basis (this is definitely a power-user feature), and could indeed be done by setting the global variable when necessary. The goal would be that -- for 99% of users -- they wouldn't need to do import { env } from '@huggingface/transformers';

Semantic clarity: An "environment" (env) should represent static values based on the environment an app runs in (e.g., IS_BROWSER_ENV, IS_FS_AVAILABLE). Most of the current env properties are actually runtime configuration options, not environment detection.

These actually aren't exposed to the user via the env export. Instead, they are defined for internal use via the apis export. This isn't accessible to users.

transformers.js/src/env.js

Lines 50 to 77 in 2ec882e

	export const apis = Object.freeze({
	/** Whether we are running in a browser environment (and not a web worker) */
	IS_BROWSER_ENV,
	/** Whether we are running in a web worker environment */
	IS_WEBWORKER_ENV,
	/** Whether the Cache API is available */
	IS_WEB_CACHE_AVAILABLE,
	/** Whether the WebGPU API is available */
	IS_WEBGPU_AVAILABLE,
	/** Whether the WebNN API is available */
	IS_WEBNN_AVAILABLE,
	/** Whether the Node.js process API is available */
	IS_PROCESS_AVAILABLE,
	/** Whether we are running in a Node.js-like environment (node, deno, bun) */
	IS_NODE_ENV,
	/** Whether the filesystem API is available */
	IS_FS_AVAILABLE,
	/** Whether the path API is available */
	IS_PATH_AVAILABLE,
	});

That said, there may be very useful cases for a per-session initialization, like cache options (similar to the cache_dir in the python library), but this wouldn't require a large refactor for it. Overall, I don't think this is a priority for v4, but I am open to discussion if there is a killer use case this would unlock! 🤗

[nico-martin]

Originally, the env property was created to help provide a useful way to access the underlying ORT env variable (now under env.backends.onnx). For ORT, there are certain global session variables, like whether to use a proxy, wasm paths, etc. And these do indeed feel necessary to remain as global/singleton values.

Yes, that makes sense. However, in my opinion, this is a weakness in the ONNX Runtime API that we shouldn't necessarily inherit. We could work around/fix this weakness at this point by storing our own set of env variables in the session and then setting them when we interact with onnxruntime.

Another consideration is alignment with the python transformers library.

True. We should stay consistent with the Transformers library, but I also think we should adjust the parts where our ecosystems differ. JavaScript, especially in browsers, has multiple packages sharing the same runtime, making global state far more problematic.

A global env is already a bit non-standard, and I fear that adding these to per-session options would be even more complicated.

I'd frame this differently: the abstraction simplifies things for users by following standard JavaScript patterns.
I don't think its complicated. Global state is what's complicated. It creates hidden dependencies and makes debugging nearly impossible when things go wrong.
It would definitely be an abstraction, but an abstraction that would allow us to move away from the "non-standard" way to a more common approach.

I don't think there are enough users who would need multiple remote host locations on a per-session basis (this is definitely a power-user feature)

That is probably true for the moment. But looking forward I hope that Transformers.js will be a standard building block for using ML in the browser. That also means it will be used (and is already used by some) packages that build on top of Transformers.js.

Here's the scenario I'm concerned about: A developer installs two packages: local-vector-db (using feature-extraction with Transformers.js) and local-translator (using translation with Transformers.js). Both packages configure their own remote hosts internally. Last one loaded wins, and suddenly one package is broken in ways that are completely invisible to the end user. They have no idea why, because the configuration conflict happens deep in dependency chains they don't control.

It might not be an issue now that we know of, but will definitely be one if Transformers.js is to become standard infrastructure for browser ML (which I hope it will). Then we need to design for composition. Real-world use cases like #1182 already demonstrate the need for per-pipeline configuration differences, even within a single application.

These actually aren't exposed to the user via the env export. Instead, they are defined for internal use via the apis export. This isn't accessible to users.

Yes, thats true and we should definitely keep internal APIs internal. But for the env parts, except "version", they dont actually depend on the environment, but rather on the session.

Bottom line: Session-based configuration would enable a package ecosystem to grow around Transformers.js. Global state works fine for direct usage, but it prevents libraries from safely building on top of it without risking conflicts.