fix(docs): update install instructions and the AWS S3 tutorial

CONTRIBUTING.md

@@ -106,7 +106,7 @@ As the react UI is embedded in the rust binary, we need to build the react app b
 
 ## Testing release (alpha, beta, rc)
 
-The following Quickwit installation command `curl -L https://install.quickwit.io | sh` always installs the latest stable version of quickwit. To make it easier in installing and testing new (alpha, beta, rc) releases, you can manually pull and execute the script as `./install.sh --allow-any-latest-version`. This will force the script to install any latest available release package.
+The following Quickwit installation command `curl -L https://raw.githubusercontent.com/quickwit-oss/quickwit/main/install.sh | sh` always installs the latest stable version of quickwit. To make it easier in installing and testing new (alpha, beta, rc) releases, you can manually pull and execute the script as `./install.sh --allow-any-latest-version`. This will force the script to install any latest available release package.
 
 ## Tracking licenses
 

distribution/docker/ubuntu/Dockerfile

@@ -1,7 +1,7 @@
 FROM ubuntu:noble@sha256:66460d557b25769b102175144d538d88219c077c678a49af4afca6fbfc1b5252 AS builder
 
 RUN apt-get update && apt-get install -y curl
-RUN curl -L https://install.quickwit.io | sh
+RUN curl -L https://raw.githubusercontent.com/quickwit-oss/quickwit/main/install.sh | sh
 
 
 FROM ubuntu:noble@sha256:66460d557b25769b102175144d538d88219c077c678a49af4afca6fbfc1b5252 AS quickwit

docs/get-started/installation.md

@@ -139,7 +139,7 @@ To easily install Quickwit on your machine, just run the command below from your
 The script detects the architecture and then downloads the correct binary archive for the machine.
 
 ```bash
-curl -L https://install.quickwit.io | sh
+curl -L https://raw.githubusercontent.com/quickwit-oss/quickwit/main/install.sh | sh
 ```
 
 All this script does is download the correct binary archive for your machine and extracts it in the current working directory. This means you can download any desired archive from [github](https://github.com/quickwit-oss/quickwit/releases) that matches your OS architecture and manually extract it anywhere.

docs/get-started/quickstart.md

@@ -14,7 +14,7 @@ The Quickwit installer automatically picks the correct binary archive for your e
 This method works only for [some OS/architectures](installation.md#download), and you will also need to install some [external dependencies](installation.md#note-on-external-dependencies).
 
 ```bash
-curl -L https://install.quickwit.io | sh
+curl -L https://raw.githubusercontent.com/quickwit-oss/quickwit/main/install.sh | sh
 ```
 
 ```bash

docs/get-started/tutorials/tutorial-hdfs-logs-distributed-search-aws-s3.md

@@ -6,6 +6,9 @@ icon_url: /img/tutorials/aws-logo.png
 sidebar_position: 6
 ---
 
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
 In this guide, we will index about 20 million log entries (7 GB decompressed) on AWS S3 using an EC2 instance and launch a three-node distributed search cluster.
 
 Example of a log entry:
@@ -34,7 +37,7 @@ First of all, let's create an EC2 instance, install a Quickwit binary, and [conf
 ## Install
 
 ```bash
-curl -L https://install.quickwit.io | sh
+curl -L https://raw.githubusercontent.com/quickwit-oss/quickwit/main/install.sh | sh
 cd quickwit-v*/
 ```
 
@@ -47,8 +50,10 @@ export S3_PATH=s3://{path/to/bucket}/indexes
 ```
 
 :::note
-You'll want to include the necessary authorization for the given bucket, this can be done by setting the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
-environment variables, or via the AWS credentials file. Usually located at `~/.aws/credentials`.
+Quickwit needs access to the bucket and the objects it contains. On EC2, attach an IAM role through an instance
+profile. Quickwit picks up those credentials automatically. The role needs `s3:ListBucket` on the bucket, plus
+`s3:GetObject`, `s3:PutObject`, `s3:DeleteObject`, `s3:ListMultipartUploadParts`, and `s3:AbortMultipartUpload` on
+its objects.
 
 For more information, check out [our AWS setup guide](../../guides/aws-setup).
 :::
@@ -60,6 +65,12 @@ Now we can create a Quickwit config file.
 echo "version: 0.8
 node_id: searcher-1
 listen_address: 0.0.0.0
+enabled_services:
+  - metastore
+  - control_plane
+  - searcher
+  - indexer
+  - janitor
 metastore_uri: ${S3_PATH}
 default_index_root_uri: ${S3_PATH}
 " > config.yaml
@@ -75,12 +86,23 @@ default_index_root_uri: ${S3_PATH}
 > default_index_root_uri: ${S3_PATH}
 >```
 
+`enabled_services` controls which services run on each node. Here `searcher-1` runs the five services listed above,
+while nodes 2 and 3 run only the searcher.
+
 We are now ready to start Quickwit.
 
 ```bash
 ./quickwit run --config config.yaml
 ```
 
+:::note
+
+`quickwit run` stays in the foreground. Keep it running and open a second connection to the same instance for the
+index and ingest commands that follow. To keep access to the node logs after disconnecting, run it under `tmux` or
+redirect its output with `nohup`.
+
+:::
+
 ## Create your index
 
 ```bash
@@ -136,7 +158,7 @@ We can now create the index with the `create` subcommand.
 :::note
 
 The `create` command sends the index configuration to the running Quickwit node. The node stores the file-backed
-metastore at `s3://path-to-your-bucket/hdfs-logs/metastore.json`.
+metastore at `s3://path-to-your-bucket/indexes/hdfs-logs/metastore.json`, under the prefix set in `S3_PATH`.
 
 To run this command from your local machine, add `--endpoint http://<quickwit-host>:7280`.
 
@@ -164,11 +186,41 @@ To run this command from your local machine, add `--endpoint http://<quickwit-ho
 :::
 
 
+:::note
+
+If ingestion fails after committing some documents, retrying appends to those documents, so the search below may
+report more than 345 hits. Reset the index before retrying:
+
+```bash
+./quickwit index clear --index hdfs-logs
+```
+
+This permanently deletes all indexed data for `hdfs-logs`.
+
+:::
+
 You can check it's working by using `search` subcommand and look for `ERROR` in `severity_text` field:
+
+<Tabs>
+
+<TabItem value="cli" label="CLI">
+
 ```bash
 ./quickwit index search --index hdfs-logs --query "severity_text:ERROR"
 ```
 
+</TabItem>
+
+<TabItem value="curl" label="cURL">
+
+```bash
+curl "http://127.0.0.1:7280/api/v1/hdfs-logs/search?query=severity_text:ERROR"
+```
+
+</TabItem>
+
+</Tabs>
+
 which returns the json
 
 ```json
@@ -205,55 +257,77 @@ Also, it needs `{rest.listen_port} + 1` for gRPC communication between instances
 
 In AWS, you can create a security group to group these inbound rules. Check out the [network section](../../guides/aws-setup) of our AWS setup guide.
 
-To make things easier, let's create a security group that opens the TCP/UDP port range [7200-7300]. 
-Next, create two additional EC2 instances using the previously created security group. Take note of each instance's public IP address.
+Create one security group for the Quickwit cluster. Set the security group itself as the source and allow TCP ports
+7280 and 7281, plus UDP port 7280.
+
+Attach this group to all three instances, including the `searcher-1` instance you launched earlier. All three nodes
+exchange cluster gossip over UDP port 7280 in both directions, and the searcher nodes communicate with each other
+over gRPC on TCP port 7281, also in both directions. Each instance must therefore accept inbound traffic from the
+other instances in the security group. Cluster communication can fail if an instance does not have this group
+attached.
+
+Next, create two additional EC2 instances using this security group, and note the private IP address of `searcher-1`.
 
-SSH into the second and third EC2 instances, install Quickwit, and [configure the environment](../../guides/aws-setup) to let Quickwit access the index S3 bucket.
+Connect to the second and third EC2 instances, install Quickwit, and
+[configure AWS access](../../guides/aws-setup) for the index bucket.
 
 ```bash
-curl -L https://install.quickwit.io | sh
+curl -L https://raw.githubusercontent.com/quickwit-oss/quickwit/main/install.sh | sh
 cd quickwit-v*/
 ```
 
 And configure the environment so instances can form a cluster:
 
 ```bash
 export S3_PATH=s3://{path/to/bucket}/indexes
-export IP_NODE_1={first-ec2-instance-public-ip}
+export IP_NODE_1={first-ec2-instance-private-ip}
 ```
 
+:::note
+
+Because all three nodes are in the same VPC, set `peer_seeds` to the private IP address of `searcher-1`. In this setup
+Quickwit advertises each node's private address, and traffic sent to a public IP does not match the self-referencing
+security group rule above.
+
+:::
+
 ```bash
 # configuration for our second node
 echo "version: 0.8
 node_id: searcher-2
 metastore_uri: ${S3_PATH}
 default_index_root_uri: ${S3_PATH}
 listen_address: 0.0.0.0
+enabled_services:
+  - searcher
 peer_seeds:
   - ${IP_NODE_1} # searcher-1
 " > config.yaml
 
 # Start a Quickwit searcher.
-./quickwit run --service searcher --config config.yaml
+./quickwit run --config config.yaml
 ```
 
 ```bash
 # configuration for our third node
 echo "version: 0.8
 node_id: searcher-3
 listen_address: 0.0.0.0
+enabled_services:
+  - searcher
 peer_seeds:
   - ${IP_NODE_1} # searcher-1
 metastore_uri: ${S3_PATH}
 default_index_root_uri: ${S3_PATH}
 " > config.yaml
 
 # Start a Quickwit searcher.
-./quickwit run --service searcher --config config.yaml
+./quickwit run --config config.yaml
 ```
 
 
-You will see in the terminal the confirmation that the instance has joined the existing cluster. Example of such a log:
+Each searcher logs that it is joining the cluster. This line is emitted before any peer has been contacted, so on its
+own it does not prove that the cluster formed. Example of such a log:
 
 ```
 2023-03-19T16:44:56.918Z  INFO quickwit_cluster::cluster: Joining cluster. cluster_id=quickwit-default-cluster node_id=searcher-2 enabled_services={Searcher} gossip_listen_addr=0.0.0.0:7280 gossip_advertise_addr=172.31.30.168:7280 grpc_advertise_addr=172.31.30.168:7281 peer_seed_addrs=172.31.91.203:7280
@@ -265,7 +339,13 @@ Now we can query one of our instance directly by issuing http requests to one of
 curl -v "http://127.0.0.1:7280/api/v1/hdfs-logs/search?query=severity_text:ERROR"
 ```
 
-Check out the logs of all instances and you will see that all nodes are working.
+To confirm that the cluster actually formed, list its members:
+
+```bash
+curl -s http://127.0.0.1:7280/api/v1/cluster
+```
+
+The `ready_nodes` array should list `searcher-1`, `searcher-2`, and `searcher-3`.
 
 ## Load balancing incoming requests
 
@@ -282,7 +362,10 @@ Let's do some cleanup by deleting the index:
 ./quickwit index delete --index hdfs-logs
 ```
 
-Also remember to remove the security group to protect your EC2 instances. You can just remove the instances if you don't need them.
+Then terminate all three EC2 instances and delete the cluster security group. Stopping the instances continues to
+incur EBS charges, and when you terminate them, delete any EBS volumes that are not set to delete on termination. The
+`index delete` command removes only the `hdfs-logs` index, so if the bucket is dedicated to this tutorial, empty and
+delete it to stop its storage charges.
 
 Congratulations! You finished this tutorial!
 

docs/get-started/tutorials/tutorial-hdfs-logs.md

@@ -33,7 +33,7 @@ Here is an example of a log entry:
 Let's download and install Quickwit.
 
 ```bash
-curl -L https://install.quickwit.io | sh
+curl -L https://raw.githubusercontent.com/quickwit-oss/quickwit/main/install.sh | sh
 cd quickwit-v*/
 ```