Batched per-db _index_info endpoint

[nickva]

Implement a new /{db}/_index_info endpoint to return the status of all the indexes in the database in a single endpoint. It matches the general pattern of _dbs_infos where we return the db info for all databases in a cluster. Just like with _db_infos, where it doesn't return anything new just a better and more optimized summary of all dbs, the new endpoint returns all the index status in a single easy to use endpoint. User could, for example, fetch all the design docs, parse them, and with an if/else check call a per-index specific _info endpoint such as /{db}/_design/{ddoc}/_info, /{db}/_design/{ddoc}/_nouveau_info/{index} etc. to get all this information, but that's not very ergonomic and it's slower.

One of the main uses of this endpoint is to find out when all the indexes in the db have been built. Each index returns an updates_pending object with the pending bounds across all copies of each range. With a minimum (the best copy), maximum (the worst copy) and a count of how many live copies returned a valid result. This information is more accurate than what we might get with the view group _info calls. It turns out for view groups _info we broke the minimum/preferred/total updates_pending stats for view groups. That happened when we switched the collection to work as a ring (the COUCHDB-3053 dedup fix), so then only the first responding copy of each range is aggregated, total = minimum and preferred only counts the responding shards. The new endpoint on the other hand, waits and aggregates stats from all copies of the index.

[ricellis]

Is this example correct?
It suggests that the API will list the same error for each search index in defined in the ddoc if search is unavailable. I suppose this is covering the case of other possible errors too? I'm trying to understand if the idea is to present the error/reason at the top-level or for each defined index.

To my mind the problem with this approach to errors is that it makes reusing existing models of the search (or view or query) index information impossible because it introduces new error and reason fields into those models (that I think are invalid in the existing case).

[nickva]

It's for the case when we couldn't get any information for any copy of the index on any shards. It would be either a cluster is very unhealthy (really partitioned) or an index type is disabled. If we get at least some copies as a response we'll return those stats with a copies values.

For the search case or nouveau case it makes sense to consider those services disabled as a real possibility. And yeah, it would list that error for any search index and for every nouveau index if we have those defined but index service is not available.

I guess we could hide those index types if the services are disabled but then we'd be hiding indexes form users and that could be confusing, too. If we do hide them if index types are disabled we'd still have to see what to return if no copies return an info object. We could crash the whole response, or maybe remove all the fields except "copies": 0 perhaps?

"ingredients": {"updates_pending": {"copies": 0}}

But that seems a kind of odd too...

[ricellis]

Thinking about it a bit more another pain point with the proposed approach is that the key for view indexes is dynamic (the ddoc name) which makes it harder to model the response type vs the static keys for search_indexes and nouveau_indexes.

How about something like:

{
    "db_name": "recipes", // instead of "name" to make it clear what name this is
    "copies_expected": 6,
    "indexes" : {
        "view_indexes": { // add this to better match the search peers and indicate the type of the nested structure
            "_design/cookbook": {
                "ok" : true, // false for error cases (presence of ok is Couch convention, omission is more normal for couch failure cases, but I think a false is preferable)
                "error" : "omit if ok",
                "reason": "omit if ok",
                "view_index": {
                    // existing model used in `GET /{db}/_design/{ddoc}/_info`
                }
            }
        },
        "search_indexes": {
            "ingredients": {
                "ok" : true,
                "error" : "omit if ok",
                "reason": "omit if ok",
                "search_index": { // add this to separate the error info from the index info and align with existing `GET /{db}/_design/{ddoc}/_search_info/{index}_search_info` naming and the view structure in this proposed endpoint
                }
            }
        },
        "nouveau_indexes": {
            "ingredients": {
                "ok" : true,
                "error" : "omit if ok",
                "reason": "omit if ok",
                "search_index": { // same idea as for search, my understanding from https://docs.couchdb.org/en/stable/api/ddoc/nouveau.html#db-design-ddoc-nouveau-info-index is that this is still called search_index for nouveau
                }
            }
        }
    }
}

I think this kind of structure would make the types from the existing info endpoints reusable under the view_index and search_index keys as well as making room for success/error information.

The existing info responses are objects pairing e.g. a name and view_index so potentially even better alignment to those would be possible by using arrays rather than dictionaries e.g.

        "view_indexes": [
            {
                "name": "_design/cookbook",
                "view_index": {
                    // existing view_index model from _info
                }
            }
        ], //etc

but that gets us back to the place where those existing info responses don't have ok/error/reason. Another way around that would be splitting success/error at a higher level e.g.

{
    "db_name": "recipes", // instead of "name" to make it clear what name this is
    "copies_expected": 6,
    "indexes" : {
        "view_indexes": [
            {
                "name": "_design/cookbook",
                "view_index": {
                    // existing view_index model from _info
                }
            }
        ],
        "search_indexes": [...],
        "nouveau_indexes": [...]
    },
    "error_indexes" : {
        "view_indexes": [
            {
                "name" : "_design/recipebook",
                "error" : "foo",
                "reason": "bar",
            }
        ],
        "search_indexes": [...],
        "nouveau_indexes": [...]
    },

Food for thought anyway.

[ricellis]

Oh I should also have said re-using the existing structures depends on updates_pending matching in both places, the existing one for view_index has minimum, preferred, total- IIUC maximum and copies are new and not renames of these existing values, but is there any reason not to include them in the existing endpoints so that the schema is the same between the *_info and this bulk version?

[nickva]

Yeah I like using arrays rather than dictionaries a bit better. Excellent point @ricellis

Oh I should also have said re-using the existing structures depends on updates_pending matching in both places, the existing one for view_index has minimum, preferred, total- IIUC maximum and copies are new and not renames of these existing values, but is there any reason not to include them in the existing endpoints so that the schema is the same between the *_info and this bulk version?

The issue is with the data they return today. It's simply broken. We don't actually wait for all the shards in those endpoints to properly report min/max values. We could fix those I suppose, too but that's a slightly bigger change.

I'll try to get a list of objects shape first going and we can see how it looks