#626: Add routing field in metadata - Solr StatusUpdaterBolt

[mvolikas]

#626 Now one can specify a routing key, which is added to the status metadata.

[jnioche]

do we need doRouting at all? Not sure how the routing is done for SOLR, is it via the collection definition or the indexing API as in OpenSearch?

[rzo1]

do we need doRouting at all? Not sure how the routing is done for SOLR, is it via the collection definition or the indexing API as in OpenSearch?

At least the variable is never used, so might be dropped.

[mvolikas]

do we need doRouting at all?

I'm aware that doRouting is not currently used. I left the variable for use in #620 but we can remove it for now.

Not sure how the routing is done for SOLR, is it via the collection definition or the indexing API as in OpenSearch?

If I'm not mistaken it depends on the router.name specified when creating the collection. I was thinking to use the compositeId option so that we do not manually specify the destination shard during indexing.

[jnioche]

do we need doRouting at all?

I'm aware that doRouting is not currently used. I left the variable for use in #620 but we can remove it for now.

yes please, let's add it back later with the rest of the code that relies on it

Not sure how the routing is done for SOLR, is it via the collection definition or the indexing API as in OpenSearch?

If I'm not mistaken it depends on the router.name specified when creating the collection. I was thinking to use the compositeId option so that we do not manually specify the destination shard during indexing.

I suppose what we need is to use

router.field (v1), router (v2)

and point it to the key field, otherwise it will take the unique document key. Is that correct?

[mvolikas]

and point it to the key field, otherwise it will take the unique document key. Is that correct?

Yes, you are right - thanks for noting!

[jnioche]

and point it to the key field, otherwise it will take the unique document key. Is that correct?

Yes, you are right - thanks for noting!

where should router.field=key be specified? in solrconfig.xml?

[mvolikas]

Thanks for the patience - I just found some time to look again into this :-)

where should router.field=key be specified? in solrconfig.xml?

I don't think this will work. The router.field is configured during the collection creation with the Collections API. (collection specific)

So, for #620 we could do the following:

• Add a new property solr.status.routing.shards: 10 to solr-conf.yaml.
• Add a script that creates the "status" collection with the numShards equal to the property above, and the router.field equal to the solr.status.routing.fieldname: "key".
• The user should make sure that the SolrSpout parallelism equals the number of shards specified.

[jnioche]

Thanks @mvolikas
Ok so the logic will be done in #620 so for this one we just need to remove SolrStatusRoutingParamName, correct?

[mvolikas]

Yes, this variable is #620 related and was unused. I removed it for this PR.

[jnioche]

Thanks @mvolikas