Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Rust supplies the native engine; C, Python, Node.js, and Go use that engine. .NE

## Quick start

Choose a language in the table above for installation commands, a working example, and its API reference. Start with [queue concepts](https://cloudtoid.com/docs/concepts/) when connecting separate processes or mixing languages.
Choose a language in the table above for installation commands, a working example, and its API reference. Start with [queue concepts](https://cloudtoid.com/docs/concepts/) when connecting separate processes or mixing languages. For a complete two-process example, follow the [Rust-to-Python shared-memory messaging tutorial](https://cloudtoid.com/docs/python-rust/).

Queues are transient: keep at least one publisher or subscriber connected throughout the handoff. Once all endpoints are gone, unread messages are lost and reopening the queue starts fresh.

Expand Down
1 change: 1 addition & 0 deletions src/website/docs/overview.html
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ <h2 id="choose-language">Choose your language</h2>
<h2 id="first-queue">Your first queue</h2>
<ol><li>Install the package for your language. Go also needs the C SDK.</li><li>Choose a short queue name and a capacity, such as <code>65536</code> bytes. On Unix, choose an explicit shared directory when processes use different runtimes.</li><li>Open a subscriber and a publisher using the same identity and capacity.</li><li>Send bytes. Check the result: a full queue needs an application-level retry or backpressure policy.</li><li>Receive bytes and close each endpoint when its work is finished.</li></ol>
<aside class="doc-note"><strong>Keep participants connected.</strong> The queue is transient. Once the last publisher or subscriber closes or exits, unread messages are lost. Reopening the same name starts a fresh queue.</aside>
<p><a href="/docs/python-rust/">Try the two-process Rust-to-Python tutorial →</a> for a complete runnable example.</p>
<h2 id="how-to-use">Which receive should I use?</h2>
<p>Use a nonblocking receive when your application already controls scheduling. Use a waiting receive when you want the library to wait for work. Reuse caller-owned buffers in Rust, Go, C, or .NET to avoid allocating a result buffer on each receive.</p>
<div class="doc-table"><table><thead><tr><th>Language</th><th>Try once</th><th>Wait for work</th></tr></thead><tbody>
Expand Down
20 changes: 13 additions & 7 deletions src/website/docs/pages.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
{
"slug": "",
"label": "Overview",
"title": "Developer documentation",
"title": "Shared-memory IPC documentation",
"description": "Build fast cross-process messaging with Cloudtoid Interprocess. Installation guides and API references for Rust, Node.js, Go, C, Python, and .NET."
},
{
Expand All @@ -11,6 +11,12 @@
"title": "Queue lifetime, delivery, and interoperability",
"description": "Understand transient queue lifetime, competing subscribers, capacity, message ordering, crash recovery, and cross-language compatibility in protocol v3."
},
{
"slug": "python-rust",
"label": "Python ↔ Rust tutorial",
"title": "Send messages from Rust to Python with shared memory",
"description": "Run a Rust publisher and Python subscriber in separate processes. A practical shared-memory IPC tutorial with installation, working code, and queue lifetime explained."
},
{
"slug": "protocol",
"label": "Protocol v3",
Expand All @@ -20,37 +26,37 @@
{
"slug": "rust",
"label": "Rust",
"title": "Rust API reference",
"title": "Rust shared-memory IPC: installation and API",
"description": "Install cloudtoid-interprocess and use Options, Publisher, and Subscriber. Reference for batch sends, reusable receive buffers, blocking waits, and errors."
},
{
"slug": "node",
"label": "Node.js",
"title": "Node.js & TypeScript API reference",
"title": "Node.js shared-memory IPC: installation and API",
"description": "Install @cloudtoid/interprocess. Send Uint8Array messages, receive Buffers, cancel with AbortSignal, and close endpoints using the Node.js API."
},
{
"slug": "go",
"label": "Go",
"title": "Go API reference",
"title": "Go shared-memory IPC: installation and API",
"description": "Use Cloudtoid Interprocess from Go with cgo. Configure Options, send and receive byte slices, reuse buffers, and cancel receives with context.Context."
},
{
"slug": "c",
"label": "C",
"title": "C API reference",
"title": "C shared-memory IPC: installation and API",
"description": "Install the Cloudtoid Interprocess C SDK. Reference for handles, status codes, receive timeouts, owned buffers, and safe shutdown."
},
{
"slug": "python",
"label": "Python",
"title": "Python API reference",
"title": "Python shared-memory IPC: installation and API",
"description": "Build Cloudtoid Interprocess for Python. Send bytes and buffer objects, receive with timeouts, use context managers, and handle queue exceptions."
},
{
"slug": "dotnet",
"label": ".NET",
"title": ".NET API reference",
"title": "C# / .NET shared-memory IPC: installation and API",
"description": "Install Cloudtoid.Interprocess from NuGet. Use QueueFactory, QueueOptions, IPublisher, and ISubscriber with reusable buffers and CancellationToken."
}
]
68 changes: 68 additions & 0 deletions src/website/docs/python-rust.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
<p class="doc-lead">Send five messages from a Rust process to a Python process on the same machine. Both programs join one shared-memory queue: no socket server or broker is needed.</p>
<p>This example uses UTF-8 text. Interprocess transports bytes without choosing a serialization format for you; you can use the same pattern for binary records or serialized application messages.</p>
<h2 id="setup">Install and prepare</h2>
<p>Use a supported 64-bit Linux, macOS, or Windows system with Python 3.9+, Rust 1.87+, Git, and a native linker. Create a working directory and a Python virtual environment, then activate it using the command for your shell:</p>
<pre><code class="language-bash">mkdir ipc-demo
cd ipc-demo
python -m venv .venv</code></pre>
<p>macOS / Linux:</p>
<pre><code class="language-bash">source .venv/bin/activate</code></pre>
<p>Windows PowerShell:</p>
<pre><code class="language-powershell">.venv\Scripts\Activate.ps1</code></pre>
<p>Use <code>python3</code> instead of <code>python</code> if that is how your system names Python 3. The Python package currently builds from source; it is not yet on PyPI.</p>
<pre><code class="language-bash">python -m pip install "git+https://github.com/cloudtoid/interprocess.git@native-v3.0.1#subdirectory=src/python"
cargo new sender
cd sender
cargo add cloudtoid-interprocess@3.0.1
cd ..</code></pre>
<h2 id="receiver">Write the Python subscriber</h2>
<p>Save this as <code>receive.py</code> in <code>ipc-demo</code>. It opens the queue before printing <code>Ready</code>, then waits up to 60 seconds for each message.</p>
<pre><code class="language-python">from pathlib import Path
from cloudtoid_interprocess import Subscriber

queue_path = Path("queue-data").resolve()
queue_path.mkdir(exist_ok=True)

with Subscriber("python-rust-demo", 65536, path=str(queue_path)) as subscriber:
print("Ready. Run the Rust sender in the second terminal.", flush=True)
for _ in range(5):
message = subscriber.receive(timeout=60.0)
if message is None:
raise TimeoutError("No message arrived within 60 seconds")
print(message.decode("utf-8"), flush=True)</code></pre>
<h2 id="sender">Write the Rust publisher</h2>
<p>Replace <code>sender/src/main.rs</code> with this program. The name, directory, and 65,536-byte capacity match the subscriber.</p>
<pre><code class="language-rust">use cloudtoid_interprocess::{Options, Publisher};

fn main() -&gt; Result&lt;(), Box&lt;dyn std::error::Error&gt;&gt; {
let path = std::env::current_dir()?.join("queue-data");
std::fs::create_dir_all(&amp;path)?;
let options = Options::new("python-rust-demo", 65536).with_path(path);
let publisher = Publisher::open(&amp;options)?;

for number in 1..=5 {
let message = format!("Hello from Rust: {number}");
publisher.try_send(message.as_bytes())?;
}
Ok(())
}</code></pre>
<p>Build it before starting the subscriber, so compilation does not consume the receive timeout:</p>
<pre><code class="language-bash">cargo build --release --manifest-path sender/Cargo.toml</code></pre>
<h2 id="run">Run two processes</h2>
<p>In the first terminal, from <code>ipc-demo</code> with the virtual environment active:</p>
<pre><code class="language-bash">python receive.py</code></pre>
<p>After <code>Ready</code> appears, open a second terminal in the <strong>same <code>ipc-demo</code> directory</strong> and run:</p>
<pre><code class="language-bash">cargo run --release --manifest-path sender/Cargo.toml</code></pre>
<p>The Python terminal prints:</p>
<pre><code class="language-plaintext">Hello from Rust: 1
Hello from Rust: 2
Hello from Rust: 3
Hello from Rust: 4
Hello from Rust: 5</code></pre>
<h2 id="lifetime">Why the subscriber starts first</h2>
<p>The queue is transient. The waiting Python subscriber keeps it alive after the Rust publisher exits. When Python closes the last endpoint, the queue ends; the next run starts fresh. Running the sender alone and then starting the subscriber will not preserve the messages.</p>
<p>On Unix, both programs must resolve <code>queue-data</code> to the same directory. Windows ignores this path and uses the queue name within the same session. Run both programs as the same user for this example. You can rerun the demo by starting the subscriber first again.</p>
<h2 id="next">Use this in your application</h2>
<p>The five small messages fit in this queue without retries. For a continuous producer, handle Rust's <code>Error::Full</code> with a bounded retry or your application's backpressure policy. Successful publication means the bytes entered the queue; it does not confirm that the other process handled them.</p>
<p>Multiple publishers and subscribers can join the same queue. Subscribers compete for messages: this is not broadcast. Keep at least one participant alive for as long as the queue is needed.</p>
<p>Continue with the <a href="/docs/rust/">Rust API</a>, <a href="/docs/python/">Python API</a>, and <a href="/docs/concepts/">queue lifetime and delivery guarantees</a>. For measured throughput and latency, see the <a href="/#performance">platform benchmarks</a>; this tutorial is a functional example, not a benchmark.</p>
1 change: 1 addition & 0 deletions src/website/docs/python.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
<h2 id="install">Install from source</h2>
<p>The Python package is not yet published on PyPI. Requires Python 3.9 or later, Git, Rust, and a native linker. Run in an activated virtual environment:</p>
<pre><code class="language-bash">python -m pip install "git+https://github.com/cloudtoid/interprocess.git@native-v3.0.1#subdirectory=src/python"</code></pre>
<p>Connecting different languages? Follow the <a href="/docs/python-rust/">Rust-to-Python messaging tutorial</a>.</p>
<h2 id="example">Send and receive</h2>
<pre><code class="language-python">from cloudtoid_interprocess import Publisher, Subscriber

Expand Down
1 change: 1 addition & 0 deletions src/website/docs/rust.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
<h2 id="install">Install</h2>
<p>Requires Rust 1.87 or later on a supported little-endian 64-bit platform.</p>
<pre><code class="language-bash">cargo add cloudtoid-interprocess</code></pre>
<p>Connecting different languages? Follow the <a href="/docs/python-rust/">Rust-to-Python messaging tutorial</a>.</p>
<h2 id="example">Send and receive</h2>
<p>This complete example keeps both endpoints alive. Separate processes use the same options; on Unix, add <code>.with_path("/absolute/shared/directory")</code> when their temporary directories differ.</p>
<pre><code class="language-rust">use cloudtoid_interprocess::{Options, Publisher, Subscriber};
Expand Down
7 changes: 5 additions & 2 deletions src/website/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,11 @@ def handle_endtag(self, tag):


pages = {p: Page(p) for p in root.rglob('*.html')}
expected = ['index.html', 'docs/index.html'] + [f'docs/{slug}/index.html' for slug in ('concepts', 'rust', 'node', 'go', 'c', 'python', 'dotnet')]
assert all(root / path in pages for path in expected), 'Missing documentation pages'
manifest = json.loads((root.parent / 'docs/pages.json').read_text())
expected = {root / 'index.html'} | {
root / 'docs' / page['slug'] / 'index.html' for page in manifest
}
assert set(pages) == expected, 'Missing or stale documentation pages'
titles, descriptions, canonicals = set(), set(), set()
for path, page in pages.items():
relative = path.relative_to(root).as_posix()
Expand Down