Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 21 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,38 +146,39 @@ stdout without touching the cluster; apply it with kubectl.

## API

The `ate-env-api` service exposes gRPC APIs over HTTP/2 (h2c) for environment lifecycle and in-actor operations, as well as HTTP endpoints for health checks and MCP.
The `ate-env-api` service exposes gRPC APIs for environment
lifecycle and in-actor operations, as well as HTTP endpoints for MCP.

### EnvironmentService

Manages the lifecycle of isolated execution environments. Requests are handled by `ate-env-api` and translated into Agent Substrate control plane operations:
Manages the lifecycle of isolated execution environments (defined in [`proto/ateenv/v1alpha/env.proto`](proto/ateenv/v1alpha/env.proto)). Requests are handled by `ate-env-api` and translated into Agent Substrate control plane operations:

| RPC | Type | Description |
| --- | --- | ----------- |
| `CreateEnvironment` | Unary | Creates and starts a new environment actor from an ActorTemplate |
| `GetEnvironment` | Unary | Retrieves environment details and status |
| `SuspendEnvironment` | Unary | Suspends and checkpoints the environment to snapshot storage |
| `DeleteEnvironment` | Unary | Deletes the environment permanently |
| RPC | Description |
| --- | ----------- |
| `CreateEnvironment` | Creates and starts a new environment actor from an ActorTemplate |
| `GetEnvironment` | Retrieves environment details and status |
| `SuspendEnvironment` | Suspends and checkpoints the environment to snapshot storage |
| `DeleteEnvironment` | Deletes the environment permanently |

### ProcessService

Manages asynchronous process execution and output streaming inside the environment container. Requests are proxied by `ate-env-api` directly to the `ate-env-guest` daemon:
Manages asynchronous process execution and output streaming inside the environment container (defined in [`proto/ateenv/v1alpha/guest.proto`](proto/ateenv/v1alpha/guest.proto)). Requests are proxied by `ate-env-api` directly to the `ate-env-guest` daemon:

| RPC | Type | Description |
| --- | --- | ----------- |
| `StartProcess` | Unary | Launches an asynchronous background process and returns a `process_id` |
| `GetProcess` | Unary | Retrieves process metadata, lifecycle status, timestamps, and exit code |
| `StreamProcessOutputs` | Server Streaming | Streams real-time `stdout` and `stderr` output chunks |
| `KillProcess` | Unary | Terminates a running background process and its child process tree |
| RPC | Description |
| --- | ----------- |
| `StartProcess` | Launches a process and returns a process ID. |
| `GetProcess` | Retrieves process metadata and status |
| `StreamProcessOutputs` | Streams stdout and stderr chunks |
| `KillProcess` | Terminates a running background process |

### FileSystemService

Provides chunked streaming file reading and writing within the environment container without unbounded memory usage. Requests are proxied by `ate-env-api` directly to the `ate-env-guest` daemon:
Provides chunked streaming file reading and writing within the environment container without unbounded memory usage (defined in [`proto/ateenv/v1alpha/guest.proto`](proto/ateenv/v1alpha/guest.proto)). Requests are proxied by `ate-env-api` directly to the `ate-env-guest` daemon:

| RPC | Type | Description |
| --- | --- | ----------- |
| `ReadFile` | Server Streaming | Streams raw binary or text file contents in chunks |
| `WriteFile` | Client Streaming | Streams raw binary or text chunks directly to a target file |
| RPC | Description |
| --- | ----------- |
| `ReadFile` | Streams raw binary or text file contents in chunks |
| `WriteFile` | Streams raw binary or text chunks directly to a target file |


## Built-in MCP Server
Expand Down
20 changes: 10 additions & 10 deletions clients/go/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (
"strings"

"github.com/agent-substrate/env/internal/ate"
ateenvv1 "github.com/agent-substrate/env/proto/ateenv/v1"
ateenvv1alpha "github.com/agent-substrate/env/proto/ateenv/v1alpha"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials/insecure"
Expand All @@ -36,9 +36,9 @@ type Client struct {
endpoint string
opts ClientOptions
grpcConn *grpc.ClientConn
grpc ateenvv1.EnvironmentServiceClient
process ateenvv1.ProcessServiceClient
filesystem ateenvv1.FileSystemServiceClient
grpc ateenvv1alpha.EnvironmentServiceClient
process ateenvv1alpha.ProcessServiceClient
filesystem ateenvv1alpha.FileSystemServiceClient
}

// NewClient returns a Client targeting endpoint.
Expand Down Expand Up @@ -70,9 +70,9 @@ func NewClient(opts ClientOptions) (*Client, error) {
endpoint: endpoint,
opts: opts,
grpcConn: grpcConn,
grpc: ateenvv1.NewEnvironmentServiceClient(grpcConn),
process: ateenvv1.NewProcessServiceClient(grpcConn),
filesystem: ateenvv1.NewFileSystemServiceClient(grpcConn),
grpc: ateenvv1alpha.NewEnvironmentServiceClient(grpcConn),
process: ateenvv1alpha.NewProcessServiceClient(grpcConn),
filesystem: ateenvv1alpha.NewFileSystemServiceClient(grpcConn),
}, nil
}

Expand All @@ -86,7 +86,7 @@ func (c *Client) Close() error {

// Create registers a new env with the parameters given in req and
// starts it using the gRPC EnvironmentService.
func (c *Client) Create(ctx context.Context, req *ateenvv1.CreateEnvironmentRequest) (*Env, error) {
func (c *Client) Create(ctx context.Context, req *ateenvv1alpha.CreateEnvironmentRequest) (*Env, error) {
resp, err := c.grpc.CreateEnvironment(ctx, req)
if err != nil {
return nil, fromGRPCError(err)
Expand All @@ -97,7 +97,7 @@ func (c *Client) Create(ctx context.Context, req *ateenvv1.CreateEnvironmentRequ

// Suspend checkpoints and stops the environment using the gRPC EnvironmentService.
func (c *Client) Suspend(ctx context.Context, atespace, id string) error {
req := &ateenvv1.SuspendEnvironmentRequest{
req := &ateenvv1alpha.SuspendEnvironmentRequest{
Id: id,
Atespace: atespace,
}
Expand All @@ -110,7 +110,7 @@ func (c *Client) Suspend(ctx context.Context, atespace, id string) error {

// Delete removes the environment permanently using the gRPC EnvironmentService.
func (c *Client) Delete(ctx context.Context, atespace, id string) error {
req := &ateenvv1.DeleteEnvironmentRequest{
req := &ateenvv1alpha.DeleteEnvironmentRequest{
Id: id,
Atespace: atespace,
}
Expand Down
12 changes: 6 additions & 6 deletions clients/go/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import (
"github.com/agent-substrate/env/internal/internaltest/fakecontrol"
"github.com/agent-substrate/env/internal/internaltest/fakerouter"
"github.com/agent-substrate/env/internal/mcp"
ateenvv1 "github.com/agent-substrate/env/proto/ateenv/v1"
ateenvv1alpha "github.com/agent-substrate/env/proto/ateenv/v1alpha"
"github.com/agent-substrate/substrate/pkg/proto/ateapipb"
"google.golang.org/grpc"
)
Expand Down Expand Up @@ -63,9 +63,9 @@ func newFixture(t *testing.T) *fixture {
t.Cleanup(service.Close)

grpcServer := grpc.NewServer()
ateenvv1.RegisterEnvironmentServiceServer(grpcServer, service)
ateenvv1.RegisterProcessServiceServer(grpcServer, service)
ateenvv1.RegisterFileSystemServiceServer(grpcServer, service)
ateenvv1alpha.RegisterEnvironmentServiceServer(grpcServer, service)
ateenvv1alpha.RegisterProcessServiceServer(grpcServer, service)
ateenvv1alpha.RegisterFileSystemServiceServer(grpcServer, service)
mux := http.NewServeMux()
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "ok\n")
Expand Down Expand Up @@ -109,9 +109,9 @@ func (f *fixture) create(t *testing.T, id string) *env.Env {

f.router.Register(id, grpcGuestServer)

sb, err := f.client.Create(t.Context(), &ateenvv1.CreateEnvironmentRequest{
sb, err := f.client.Create(t.Context(), &ateenvv1alpha.CreateEnvironmentRequest{
Id: id,
Template: &ateenvv1.Template{
Template: &ateenvv1alpha.Template{
Name: "default-env",
Atespace: "envs",
},
Expand Down
20 changes: 10 additions & 10 deletions clients/go/env.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"io/fs"
"time"

ateenvv1 "github.com/agent-substrate/env/proto/ateenv/v1"
ateenvv1alpha "github.com/agent-substrate/env/proto/ateenv/v1alpha"
"google.golang.org/grpc/metadata"
)

Expand Down Expand Up @@ -41,15 +41,15 @@ func (e *Env) Delete(ctx context.Context) error {
// Shell runs a shell command line inside the environment using ProcessService.
func (e *Env) Shell(ctx context.Context, commandLine string) (*ShellResponse, error) {
ctx = e.withEnv(ctx)
startResp, err := e.client.process.StartProcess(ctx, &ateenvv1.StartProcessRequest{
startResp, err := e.client.process.StartProcess(ctx, &ateenvv1alpha.StartProcessRequest{
Command: []string{"sh", "-c", commandLine},
})
if err != nil {
return nil, fromGRPCError(err)
}

pid := startResp.GetProcessId()
outStream, err := e.client.process.StreamProcessOutputs(ctx, &ateenvv1.StreamProcessOutputsRequest{
outStream, err := e.client.process.StreamProcessOutputs(ctx, &ateenvv1alpha.StreamProcessOutputsRequest{
ProcessId: pid,
Follow: true,
})
Expand All @@ -67,23 +67,23 @@ func (e *Env) Shell(ctx context.Context, commandLine string) (*ShellResponse, er
return nil, fromGRPCError(err)
}
switch chunk.GetSource() {
case ateenvv1.OutputSource_OUTPUT_SOURCE_STDOUT:
case ateenvv1alpha.OutputSource_OUTPUT_SOURCE_STDOUT:
stdoutBuf.Write(chunk.GetData())
case ateenvv1.OutputSource_OUTPUT_SOURCE_STDERR:
case ateenvv1alpha.OutputSource_OUTPUT_SOURCE_STDERR:
stderrBuf.Write(chunk.GetData())
}
}

// Retrieve final process state / exit code
var exitCode int
for {
proc, err := e.client.process.GetProcess(ctx, &ateenvv1.GetProcessRequest{
proc, err := e.client.process.GetProcess(ctx, &ateenvv1alpha.GetProcessRequest{
ProcessId: pid,
})
if err != nil {
return nil, fromGRPCError(err)
}
if proc.GetStatus() != ateenvv1.ProcessStatus_PROCESS_STATUS_RUNNING {
if proc.GetStatus() != ateenvv1alpha.ProcessStatus_PROCESS_STATUS_RUNNING {
exitCode = int(proc.GetExitCode())
break
}
Expand All @@ -105,7 +105,7 @@ func (e *Env) Shell(ctx context.Context, commandLine string) (*ShellResponse, er
// The caller must close the returned reader.
func (e *Env) ReadFile(ctx context.Context, p string) (io.ReadCloser, error) {
ctx = e.withEnv(ctx)
stream, err := e.client.filesystem.ReadFile(ctx, &ateenvv1.ReadFileRequest{
stream, err := e.client.filesystem.ReadFile(ctx, &ateenvv1alpha.ReadFileRequest{
Path: p,
})
if err != nil {
Expand Down Expand Up @@ -165,7 +165,7 @@ func (e *Env) WriteFile(ctx context.Context, p string, r io.Reader, mode fs.File
return fmt.Errorf("env: reading data for %q: %w", p, readErr)
}

firstReq := &ateenvv1.WriteFileRequest{
firstReq := &ateenvv1alpha.WriteFileRequest{
Path: p,
Mode: uint32(mode.Perm()),
Chunk: buf[:n],
Expand All @@ -178,7 +178,7 @@ func (e *Env) WriteFile(ctx context.Context, p string, r io.Reader, mode fs.File
for {
n, err := r.Read(buf)
if n > 0 {
if sendErr := stream.Send(&ateenvv1.WriteFileRequest{Chunk: buf[:n]}); sendErr != nil {
if sendErr := stream.Send(&ateenvv1alpha.WriteFileRequest{Chunk: buf[:n]}); sendErr != nil {
return fromGRPCError(sendErr)
}
}
Expand Down
6 changes: 3 additions & 3 deletions clients/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,14 @@ plain gRPC (h2c). Behind that endpoint there are two distinct paths:
```

**Lifecycle path.** `Client.create/get/suspend/delete` call
`EnvironmentService` (defined in [`proto/ateenv/v1/env.proto`](../../proto/ateenv/v1/env.proto)).
`EnvironmentService` (defined in [`proto/ateenv/v1alpha/env.proto`](../../proto/ateenv/v1alpha/env.proto)).
These RPCs terminate at `ate-env-api`, which translates them into Substrate
control-plane operations: creating an actor from an ActorTemplate,
reading its status, checkpointing it to a snapshot, deleting it.

**Guest path.** Everything on an `Env` handle that executes *inside* the
environment — processes and files — calls `ProcessService` and
`FileSystemService` (defined in [`proto/ateenv/v1/guest.proto`](../../proto/ateenv/v1/guest.proto)).
`FileSystemService` (defined in [`proto/ateenv/v1alpha/guest.proto`](../../proto/ateenv/v1alpha/guest.proto)).
The client attaches `x-env-id` / `x-env-atespace` gRPC metadata to each of
these calls; `ate-env-api` uses that metadata to dial the atenet router
with the authority `<id>.<atespace>.<host-suffix>`, and the router carries
Expand Down Expand Up @@ -305,7 +305,7 @@ python3 -m venv .venv
### Regenerating gRPC stubs

Generated code under `src/ate_env/_gen/` is committed. After changing
`proto/ateenv/v1/*.proto`, regenerate from the repo root:
`proto/ateenv/v1alpha/*.proto`, regenerate from the repo root:

```bash
./clients/python/scripts/gen-protos.sh # or: make python-protos
Expand Down
14 changes: 7 additions & 7 deletions clients/python/scripts/gen-protos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,20 @@ cd "${ROOT}"
PYTHON="${PYTHON:-python3}"
OUT_DIR="clients/python/src/ate_env/_gen"

# -I proto (not proto/ateenv/v1) so descriptors register under the
# collision-safe filenames ateenv/v1/{env,guest}.proto.
# -I proto (not proto/ateenv/v1alpha) so descriptors register under the
# collision-safe filenames ateenv/v1alpha/{env,guest}.proto.
"${PYTHON}" -m grpc_tools.protoc \
-I proto \
--python_out="${OUT_DIR}" \
--pyi_out="${OUT_DIR}" \
--grpc_python_out="${OUT_DIR}" \
proto/ateenv/v1/env.proto \
proto/ateenv/v1/guest.proto
proto/ateenv/v1alpha/env.proto \
proto/ateenv/v1alpha/guest.proto

# protoc emits `from ateenv.v1 import env_pb2 as ...` in *_pb2_grpc.py, which
# protoc emits `from ateenv.v1alpha import env_pb2 as ...` in *_pb2_grpc.py, which
# does not resolve inside the _gen package; rewrite to relative imports.
# sed -i with a backup suffix is the only form portable across GNU and BSD sed.
for f in "${OUT_DIR}"/ateenv/v1/*_pb2_grpc.py; do
sed -i.bak 's/^from ateenv\.v1 import \(.*\)$/from . import \1/' "${f}"
for f in "${OUT_DIR}"/ateenv/v1alpha/*_pb2_grpc.py; do
sed -i.bak 's/^from ateenv\.v1alpha import \(.*\)$/from . import \1/' "${f}"
rm -f "${f}.bak"
done
1 change: 0 additions & 1 deletion clients/python/src/ate_env/_gen/ateenv/v1/__init__.py

This file was deleted.

49 changes: 0 additions & 49 deletions clients/python/src/ate_env/_gen/ateenv/v1/env_pb2.py

This file was deleted.

Loading