[ArnavAgrawal03]

Ingestion workers currently check Redis every two seconds. With one worker and short text documents, this leaves the worker idle between jobs even when embedding finishes in about half a second.

Add [worker].poll_delay to morphik.toml, validate that it is positive and finite, and pass it to ARQ. The default stays at 2 seconds for existing deployments. Setting poll_delay = 0.1 reduces queue latency without increasing arq_max_jobs or the number of worker processes. The tradeoff is more frequent Redis reads while idle.

Validation:

• Seven configuration tests cover existing configs, a missing worker section, the override, and zero/negative/infinite/NaN values.
• Black and Ruff pass on the changed Python files.
• Real Azure DevOps to Morphik runs used 20 disposable tickets, 20 comments, six text attachments, real Bedrock embeddings, and one ingestion worker with arq_max_jobs = 1.
• With connector 1.2.7, ten-ticket updates took 10.5s and 9.9s at 0.1s queue polling, versus 23.5s in the preceding 2s run. First-time ingestion of 26 documents took 21.3s versus 56.7s.
• All documents completed; hashes matched the manifest, ticket revisions matched Azure, and vector retrieval succeeded.

The real runs used the pinned official Core image main-ce8fece7114db60d1d0f656f2320160de2514bea with the equivalent config/WorkerSettings change applied. Configuration unit tests run against this branch. This measures short text documents; it does not establish OCR/video throughput or large-manifest performance.

After upgrading Core, opt in with:

[worker]
arq_max_jobs = 1
poll_delay = 0.1

Restart the ingestion worker after changing the setting.

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}