diff --git a/README.md b/README.md index 92431a2..c4c411f 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ Rust supplies the native engine; C, Python, Node.js, and Go use that engine. .NE ## Quick start -Choose a language in the table above for installation commands, a working example, and its API reference. Start with [queue concepts](https://cloudtoid.com/docs/concepts/) when connecting separate processes or mixing languages. +Choose a language in the table above for installation commands, a working example, and its API reference. Start with [queue concepts](https://cloudtoid.com/docs/concepts/) when connecting separate processes or mixing languages. For a complete two-process example, follow the [Rust-to-Python shared-memory messaging tutorial](https://cloudtoid.com/docs/python-rust/). Queues are transient: keep at least one publisher or subscriber connected throughout the handoff. Once all endpoints are gone, unread messages are lost and reopening the queue starts fresh. diff --git a/src/website/docs/overview.html b/src/website/docs/overview.html index 83eaf2d..ca91889 100644 --- a/src/website/docs/overview.html +++ b/src/website/docs/overview.html @@ -11,6 +11,7 @@

Choose your language

Your first queue

  1. Install the package for your language. Go also needs the C SDK.
  2. Choose a short queue name and a capacity, such as 65536 bytes. On Unix, choose an explicit shared directory when processes use different runtimes.
  3. Open a subscriber and a publisher using the same identity and capacity.
  4. Send bytes. Check the result: a full queue needs an application-level retry or backpressure policy.
  5. Receive bytes and close each endpoint when its work is finished.
+

Try the two-process Rust-to-Python tutorial → for a complete runnable example.

Which receive should I use?

Use a nonblocking receive when your application already controls scheduling. Use a waiting receive when you want the library to wait for work. Reuse caller-owned buffers in Rust, Go, C, or .NET to avoid allocating a result buffer on each receive.

diff --git a/src/website/docs/pages.json b/src/website/docs/pages.json index 6b03ac3..07a37b3 100644 --- a/src/website/docs/pages.json +++ b/src/website/docs/pages.json @@ -2,7 +2,7 @@ { "slug": "", "label": "Overview", - "title": "Developer documentation", + "title": "Shared-memory IPC documentation", "description": "Build fast cross-process messaging with Cloudtoid Interprocess. Installation guides and API references for Rust, Node.js, Go, C, Python, and .NET." }, { @@ -11,6 +11,12 @@ "title": "Queue lifetime, delivery, and interoperability", "description": "Understand transient queue lifetime, competing subscribers, capacity, message ordering, crash recovery, and cross-language compatibility in protocol v3." }, + { + "slug": "python-rust", + "label": "Python ↔ Rust tutorial", + "title": "Send messages from Rust to Python with shared memory", + "description": "Run a Rust publisher and Python subscriber in separate processes. A practical shared-memory IPC tutorial with installation, working code, and queue lifetime explained." + }, { "slug": "protocol", "label": "Protocol v3", @@ -20,37 +26,37 @@ { "slug": "rust", "label": "Rust", - "title": "Rust API reference", + "title": "Rust shared-memory IPC: installation and API", "description": "Install cloudtoid-interprocess and use Options, Publisher, and Subscriber. Reference for batch sends, reusable receive buffers, blocking waits, and errors." }, { "slug": "node", "label": "Node.js", - "title": "Node.js & TypeScript API reference", + "title": "Node.js shared-memory IPC: installation and API", "description": "Install @cloudtoid/interprocess. Send Uint8Array messages, receive Buffers, cancel with AbortSignal, and close endpoints using the Node.js API." }, { "slug": "go", "label": "Go", - "title": "Go API reference", + "title": "Go shared-memory IPC: installation and API", "description": "Use Cloudtoid Interprocess from Go with cgo. Configure Options, send and receive byte slices, reuse buffers, and cancel receives with context.Context." }, { "slug": "c", "label": "C", - "title": "C API reference", + "title": "C shared-memory IPC: installation and API", "description": "Install the Cloudtoid Interprocess C SDK. Reference for handles, status codes, receive timeouts, owned buffers, and safe shutdown." }, { "slug": "python", "label": "Python", - "title": "Python API reference", + "title": "Python shared-memory IPC: installation and API", "description": "Build Cloudtoid Interprocess for Python. Send bytes and buffer objects, receive with timeouts, use context managers, and handle queue exceptions." }, { "slug": "dotnet", "label": ".NET", - "title": ".NET API reference", + "title": "C# / .NET shared-memory IPC: installation and API", "description": "Install Cloudtoid.Interprocess from NuGet. Use QueueFactory, QueueOptions, IPublisher, and ISubscriber with reusable buffers and CancellationToken." } ] diff --git a/src/website/docs/python-rust.html b/src/website/docs/python-rust.html new file mode 100644 index 0000000..1572182 --- /dev/null +++ b/src/website/docs/python-rust.html @@ -0,0 +1,68 @@ +

Send five messages from a Rust process to a Python process on the same machine. Both programs join one shared-memory queue: no socket server or broker is needed.

+

This example uses UTF-8 text. Interprocess transports bytes without choosing a serialization format for you; you can use the same pattern for binary records or serialized application messages.

+

Install and prepare

+

Use a supported 64-bit Linux, macOS, or Windows system with Python 3.9+, Rust 1.87+, Git, and a native linker. Create a working directory and a Python virtual environment, then activate it using the command for your shell:

+
mkdir ipc-demo
+cd ipc-demo
+python -m venv .venv
+

macOS / Linux:

+
source .venv/bin/activate
+

Windows PowerShell:

+
.venv\Scripts\Activate.ps1
+

Use python3 instead of python if that is how your system names Python 3. The Python package currently builds from source; it is not yet on PyPI.

+
python -m pip install "git+https://github.com/cloudtoid/interprocess.git@native-v3.0.1#subdirectory=src/python"
+cargo new sender
+cd sender
+cargo add cloudtoid-interprocess@3.0.1
+cd ..
+

Write the Python subscriber

+

Save this as receive.py in ipc-demo. It opens the queue before printing Ready, then waits up to 60 seconds for each message.

+
from pathlib import Path
+from cloudtoid_interprocess import Subscriber
+
+queue_path = Path("queue-data").resolve()
+queue_path.mkdir(exist_ok=True)
+
+with Subscriber("python-rust-demo", 65536, path=str(queue_path)) as subscriber:
+    print("Ready. Run the Rust sender in the second terminal.", flush=True)
+    for _ in range(5):
+        message = subscriber.receive(timeout=60.0)
+        if message is None:
+            raise TimeoutError("No message arrived within 60 seconds")
+        print(message.decode("utf-8"), flush=True)
+

Write the Rust publisher

+

Replace sender/src/main.rs with this program. The name, directory, and 65,536-byte capacity match the subscriber.

+
use cloudtoid_interprocess::{Options, Publisher};
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let path = std::env::current_dir()?.join("queue-data");
+    std::fs::create_dir_all(&path)?;
+    let options = Options::new("python-rust-demo", 65536).with_path(path);
+    let publisher = Publisher::open(&options)?;
+
+    for number in 1..=5 {
+        let message = format!("Hello from Rust: {number}");
+        publisher.try_send(message.as_bytes())?;
+    }
+    Ok(())
+}
+

Build it before starting the subscriber, so compilation does not consume the receive timeout:

+
cargo build --release --manifest-path sender/Cargo.toml
+

Run two processes

+

In the first terminal, from ipc-demo with the virtual environment active:

+
python receive.py
+

After Ready appears, open a second terminal in the same ipc-demo directory and run:

+
cargo run --release --manifest-path sender/Cargo.toml
+

The Python terminal prints:

+
Hello from Rust: 1
+Hello from Rust: 2
+Hello from Rust: 3
+Hello from Rust: 4
+Hello from Rust: 5
+

Why the subscriber starts first

+

The queue is transient. The waiting Python subscriber keeps it alive after the Rust publisher exits. When Python closes the last endpoint, the queue ends; the next run starts fresh. Running the sender alone and then starting the subscriber will not preserve the messages.

+

On Unix, both programs must resolve queue-data to the same directory. Windows ignores this path and uses the queue name within the same session. Run both programs as the same user for this example. You can rerun the demo by starting the subscriber first again.

+

Use this in your application

+

The five small messages fit in this queue without retries. For a continuous producer, handle Rust's Error::Full with a bounded retry or your application's backpressure policy. Successful publication means the bytes entered the queue; it does not confirm that the other process handled them.

+

Multiple publishers and subscribers can join the same queue. Subscribers compete for messages: this is not broadcast. Keep at least one participant alive for as long as the queue is needed.

+

Continue with the Rust API, Python API, and queue lifetime and delivery guarantees. For measured throughput and latency, see the platform benchmarks; this tutorial is a functional example, not a benchmark.

diff --git a/src/website/docs/python.html b/src/website/docs/python.html index ea9f621..428db3c 100644 --- a/src/website/docs/python.html +++ b/src/website/docs/python.html @@ -3,6 +3,7 @@

Install from source

The Python package is not yet published on PyPI. Requires Python 3.9 or later, Git, Rust, and a native linker. Run in an activated virtual environment:

python -m pip install "git+https://github.com/cloudtoid/interprocess.git@native-v3.0.1#subdirectory=src/python"
+

Connecting different languages? Follow the Rust-to-Python messaging tutorial.

Send and receive

from cloudtoid_interprocess import Publisher, Subscriber
 
diff --git a/src/website/docs/rust.html b/src/website/docs/rust.html
index 2c5053e..dbebb1a 100644
--- a/src/website/docs/rust.html
+++ b/src/website/docs/rust.html
@@ -3,6 +3,7 @@
 

Install

Requires Rust 1.87 or later on a supported little-endian 64-bit platform.

cargo add cloudtoid-interprocess
+

Connecting different languages? Follow the Rust-to-Python messaging tutorial.

Send and receive

This complete example keeps both endpoints alive. Separate processes use the same options; on Unix, add .with_path("/absolute/shared/directory") when their temporary directories differ.

use cloudtoid_interprocess::{Options, Publisher, Subscriber};
diff --git a/src/website/validate.py b/src/website/validate.py
index 5035aa1..b2e8eb4 100644
--- a/src/website/validate.py
+++ b/src/website/validate.py
@@ -55,8 +55,11 @@ def handle_endtag(self, tag):
 
 
 pages = {p: Page(p) for p in root.rglob('*.html')}
-expected = ['index.html', 'docs/index.html'] + [f'docs/{slug}/index.html' for slug in ('concepts', 'rust', 'node', 'go', 'c', 'python', 'dotnet')]
-assert all(root / path in pages for path in expected), 'Missing documentation pages'
+manifest = json.loads((root.parent / 'docs/pages.json').read_text())
+expected = {root / 'index.html'} | {
+    root / 'docs' / page['slug'] / 'index.html' for page in manifest
+}
+assert set(pages) == expected, 'Missing or stale documentation pages'
 titles, descriptions, canonicals = set(), set(), set()
 for path, page in pages.items():
     relative = path.relative_to(root).as_posix()
LanguageTry onceWait for work