diff --git a/site/src/docs/examples.md b/site/src/docs/examples.md index 9fc0ae07..06e22647 100644 --- a/site/src/docs/examples.md +++ b/site/src/docs/examples.md @@ -92,6 +92,13 @@ AZURE_OPENAI_ENDPOINT=... AZURE_OPENAI_API_KEY=... ``` +The examples also work against any OpenAI-compatible gateway, such as [OrcaRouter](https://www.orcarouter.ai), by pointing `OPENAI_ENDPOINT` at the gateway's Chat Completions URL in addition to the OpenAI variables above: + +``` +# Optional: use an OpenAI-compatible gateway instead of api.openai.com +OPENAI_ENDPOINT=https://api.orcarouter.ai/v1/chat/completions +``` + ## Step 4: Run the examples Examples can be found in the `typescript/examples` directory. diff --git a/site/src/docs/python/basic-usage.md b/site/src/docs/python/basic-usage.md index de556fdf..e14ff220 100644 --- a/site/src/docs/python/basic-usage.md +++ b/site/src/docs/python/basic-usage.md @@ -83,7 +83,7 @@ def create_azure_openai_language_model(api_key: str, endpoint: str): ... For even more convenience, TypeChat also provides a function to infer whether you're using OpenAI or Azure OpenAI. -```ts +```py def create_language_model( vals: dict[str, str | None] ) -> TypeChatLanguageModel: ... @@ -94,6 +94,29 @@ pass them in. Based on whether `OPENAI_API_KEY` or `AZURE_OPENAI_API_KEY` is set, you'll get a model of the appropriate type. +#### Using OpenAI-compatible gateways + +TypeChat works with any service that exposes the OpenAI Chat Completions API. +To use a gateway such as [OrcaRouter](https://www.orcarouter.ai) or OpenRouter, +point `create_openai_language_model` at the gateway's endpoint via the +`endpoint` argument: + +```py +model = create_openai_language_model( + api_key, model, endpoint="https://api.orcarouter.ai/v1/chat/completions" +) +``` + +With `create_language_model`, set the `OPENAI_ENDPOINT` environment variable to +the gateway's Chat Completions URL alongside `OPENAI_API_KEY` and +`OPENAI_MODEL`: + +``` +OPENAI_API_KEY= +OPENAI_MODEL= +OPENAI_ENDPOINT=https://api.orcarouter.ai/v1/chat/completions +``` + The `TypeChatLanguageModel` returned by these functions has a few writable attributes you might find useful: diff --git a/site/src/docs/typescript/basic-usage.md b/site/src/docs/typescript/basic-usage.md index 9bf343b2..b05e4fb9 100644 --- a/site/src/docs/typescript/basic-usage.md +++ b/site/src/docs/typescript/basic-usage.md @@ -68,7 +68,7 @@ For convenience, TypeChat provides two functions out of the box to connect to th You can call these directly. ```ts -export function createOpenAILanguageModel(apiKey: string, model: string, endPoint? string): TypeChatLanguageModel; +export function createOpenAILanguageModel(apiKey: string, model: string, endPoint?: string): TypeChatLanguageModel; export function createAzureOpenAILanguageModel(apiKey: string, endPoint: string): TypeChatLanguageModel; ``` @@ -82,6 +82,27 @@ export function createLanguageModel(env: Record): Ty With `createLanguageModel`, you can populate your environment variables and pass them in. Based on whether `OPENAI_API_KEY` or `AZURE_OPENAI_API_KEY` is set, you'll get a model of the appropriate type. +#### Using OpenAI-compatible gateways + +TypeChat works with any service that exposes the OpenAI Chat Completions API. +To use a gateway such as [OrcaRouter](https://www.orcarouter.ai) or OpenRouter, +point `createOpenAILanguageModel` at the gateway's endpoint via the `endPoint` +argument: + +```ts +const model = createOpenAILanguageModel(apiKey, model, "https://api.orcarouter.ai/v1/chat/completions"); +``` + +With `createLanguageModel`, set the `OPENAI_ENDPOINT` environment variable to +the gateway's Chat Completions URL alongside `OPENAI_API_KEY` and +`OPENAI_MODEL`: + +``` +OPENAI_API_KEY= +OPENAI_MODEL= +OPENAI_ENDPOINT=https://api.orcarouter.ai/v1/chat/completions +``` + Regardless, of how you decide to construct your model, it is important to avoid committing credentials directly in source. One way to make this work between production and development environments is to use a `.env` file in development, and specify that `.env` in your `.gitignore`. You can use a library like [`dotenv`](https://www.npmjs.com/package/dotenv) to help load these up.