N/llm Module Adds Streamed Text Generation via generateTextStreamed

What changed

The N/llm module now exposes llm.generateTextStreamed(options) (aliased as llm.chatStreamed(options)), a streaming counterpart to the existing llm.generateText(options). Instead of waiting for the full LLM response, you can iterate over tokens as they arrive and read the partial response at any point via StreamedResponse.text.

Key details

• Governance: 100 units per call.
• Script types: Server scripts only (Scheduled, Map/Reduce, Suitelet, User Event, RESTlet, etc.).
• Default model: Cohere Command A (cohere.command-a-03-2025) when options.modelFamily is omitted. Set via the llm.ModelFamily enum.
• Default timeout: 30,000 ms. Configurable with options.timeout.
• Parallelism cap: Maximum 5 concurrent requests to the LLM per script execution; exceeding this throws MAXIMUM_PARALLEL_REQUESTS_LIMIT_EXCEEDED.
• Safety mode: Defaults to llm.SafetyMode.STRICT. Cohere models only.

Streaming pattern

The returned llm.StreamedResponse object supports the standard SuiteScript iterator protocol:

var response = llm.generateTextStreamed({
  prompt: "Summarize open sales orders",
  modelFamily: llm.ModelFamily.COHERE_COMMAND,
  modelParameters: { maxTokens: 1000, temperature: 0.2 }
});
var iter = response.iterator();
iter.each(function(token) {
  log.debug("token: " + token.value);
  log.debug("partial: " + response.text);
  return true;
});

OCI unlimited-usage mode

You can supply your own Oracle Cloud Infrastructure (OCI) Generative AI credentials via options.ociConfig to bypass NetSuite's bundled usage limits. The config accepts userId, tenancyId, compartmentId, fingerprint, and privateKey. The fingerprint and privateKey values must be NetSuite API secrets (created via Setup > Company > API Secrets); raw strings throw ONLY_API_SECRET_IS_ACCEPTED. These can also be set account-wide on the AI Preferences page > SuiteScript subtab; per-call values override the page-level defaults.

Tool calling (2025.2)

Two parameters were added in 2025.2:

• options.tools — an array of llm.Tool objects describing functions the LLM may request.
• options.toolResults — an array of llm.ToolResult objects returned after your script executes the requested tool calls. When toolResults is provided, options.prompt is ignored and the LLM generates a follow-up response based solely on the tool output.

Tool call requests appear in StreamedResponse.toolCalls. This enables an agentic loop: call generateTextStreamed with tools, inspect toolCalls, execute them, then call again with toolResults.

RAG via documents (Cohere only)

The options.documents parameter accepts an array of llm.Document objects (created with llm.createDocument(options)) for retrieval-augmented generation. Document IDs must be unique or DOCUMENT_IDS_MUST_BE_UNIQUE is thrown. This parameter is not supported for non-Cohere model families.

Model parameters

Standard LLM tuning knobs are available under options.modelParameters: maxTokens, temperature, topK, topP, frequencyPenalty, and presencePenalty. Note that for COHERE_COMMAND, you cannot set both frequencyPenalty and presencePenalty above zero simultaneously — doing so throws MUTUALLY_EXCLUSIVE_ARGUMENTS.

What to do

1. Choose streaming vs. blocking: Use generateTextStreamed when you need progressive output (e.g., writing partial results to a record or log during long generations). Use generateText when you only need the final result.
2. Budget governance: Each call costs 100 units. In Map/Reduce scripts, plan your usage within the 1,000-unit-per-invocation limit — you get at most 10 LLM calls per map or reduce invocation before needing to yield.
3. Store OCI credentials as secrets: If using unlimited-usage mode, create API secrets for your OCI private key and fingerprint before writing any script code. Raw credential strings will be rejected at runtime.
4. Adopt tool calling for agentic patterns (2025.2+): If you are on 2025.2 or later, you can define tools and implement the tool-call loop. Verify your account is on 2025.2 before using options.tools or options.toolResults — these parameters do not exist in 2025.1 and will be silently ignored or throw errors.
5. Handle content filtering: Wrap calls in try/catch for INAPPROPRIATE_CONTENT_DETECTED. The default safety mode is STRICT; if you need less aggressive filtering, explicitly set options.safetyMode (Cohere models only).
6. Respect the 5-request concurrency limit: If you are firing multiple LLM calls in parallel (e.g., via Promise.all in SuiteScript 2.1), keep concurrent requests at or below 5.