fix(docs): update install instructions and the AWS S3 tutorial - #6715
Open
dayaffe wants to merge 1 commit into
Open
fix(docs): update install instructions and the AWS S3 tutorial#6715dayaffe wants to merge 1 commit into
dayaffe wants to merge 1 commit into
Conversation
install.quickwit.io no longer resolves, which breaks the very first command in the quickstart, the installation guide, and both HDFS tutorials. The install.sh script itself is still present and maintained at the repo root, so point the docs at its raw GitHub URL rather than replacing it. That keeps the script's platform detection and latest-version resolution, and adds no version literals to maintain. This also fixes distribution/docker/ubuntu/Dockerfile, which cannot build today. Because a shell pipeline reports only the status of its last command, "curl <dead host> | sh" exits 0 and installs nothing, so the build instead fails further down at the COPY of /quickwit-v*/quickwit with a misleading error. That Dockerfile is not referenced by any workflow, which is why this went unnoticed. Published images are unaffected because they compile from source. The remaining changes fix the AWS S3 tutorial, verified end to end against a live three-node cluster on EC2: - Attach the cluster security group to the first node as well. Without this the pre-existing node drops inbound gossip and the cluster never forms. - Use the private IP of searcher-1 for peer_seeds. Quickwit advertises its private address, so a public seed cannot match a rule scoped to the security group. - Narrow the security group to TCP 7280-7281 and UDP 7280 with the group itself as the source, rather than the 7200-7300 range. - Declare enabled_services in all three node configs, so searcher-1 runs the metastore, control plane, indexer, searcher and janitor while the other two run only the searcher. This replaces the --service searcher flag on the command line. - Correct the metastore path to include the prefix set in S3_PATH. - Verify cluster formation through /api/v1/cluster, naming ready_nodes, and note that the joining cluster log line is emitted before any peer is contacted. - Offer the search step as CLI or cURL through tabs, matching the hdfs-logs tutorial. - Document recovery from a partial ingest using index clear, since re-running the ingest appends and inflates the expected 345 hits. - Use an IAM instance profile instead of static access keys on EC2. - Note that quickwit run holds the foreground and needs a second terminal. - Expand cleanup to cover instances, root volumes, the security group, and the bucket. The config format version 0.8 is intentionally left unchanged, as it is independent of the 0.9.0 binary version. Both control_plane and control-plane parse correctly in enabled_services, so the underscore here matches the node configuration reference. Closes #6660 Closes #4004
guilload
approved these changes
Aug 21, 2026
guilload
left a comment
guilload
left a comment
Member
There was a problem hiding this comment.
Nice improvements. You may want to improve the paragraph I mentioned and then merge.
|
|
||
| :::note | ||
|
|
||
| If ingestion fails after committing some documents, retrying appends to those documents, so the search below may |
Member
There was a problem hiding this comment.
@nadav-govari another instance of ingest v2 not opening new shards fast enough.
| Create one security group for the Quickwit cluster. Set the security group itself as the source and allow TCP ports | ||
| 7280 and 7281, plus UDP port 7280. | ||
|
|
||
| Attach this group to all three instances, including the `searcher-1` instance you launched earlier. Nodes 2 and 3 |
Member
There was a problem hiding this comment.
All the nodes are going to gossip in together via 7280 UDP in both directions. In addition, all the searchers are going to connect to each other in both directions too via TCP 7281 (gRPC). However this paragraph can let the reader think that all the traffic is "one-directional" from 2 and 3 to 1 and that seems misleading. At least, that how I read it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
install.quickwit.iowith a direct link in the quickstart, installation guide, both HDFS tutorials, anddistribution/docker/ubuntu/DockerfileAlso update the AWS S3 tutorial:
searcher-1as wellsearcher-1forpeer_seedsenabled_servicesin all three node configs instead of the--service searcherflagS3_PATH/api/v1/clusterindex clearrecovery from a partial ingestquickwit runholds the foregroundHow was this PR tested?
Closes #6660
Closes #4004