unhelpful 400 responses when queryFacets fails

[alaniwi]

I tried using search when some of the shards were not started, and if I included any parameters in the query string (such as project or mip_era) it would return a 400 (bad query). The underlying cause was the shards that were not started, rather the query URL itself, so a 400 is misleading.

It turns out that the queryFacets is doing a round-robin of shards querying the luke URL -- at

esg-search/src/java/main/esg/search/query/impl/solr/LukeHandlerFacetProfileImpl.java

Lines 84 to 103 in 43fb32f

	for (String shard : shards) {
	final String fullUrl = "http://" + shard + "/datasets/admin/luke/?numTerms=0";
	if (LOG.isInfoEnabled()) LOG.info("Querying all available facets from URL="+fullUrl);
	String response = httpClient.doGet(new URL(fullUrl));
	final Document doc = xmlParser.parseString(response);
	xPath = XPath.newInstance(XPATH);
	for (final Object obj : xPath.selectNodes(doc)) {
	String facetKey = ((Element)obj).getAttributeValue("name");
	if (!_facets.containsKey(facetKey)) { // already counted
	// avoid faceting on fields that have too many values to improve performance
	if (!QueryParameters.NOT_FACETS.contains(facetKey)) {
	_facets.put(facetKey, new FacetImpl(facetKey, facetKey, "")); // facet key = facet name
	if (LOG.isInfoEnabled()) LOG.info("Using facet:"+facetKey);
	}
	}
	}
	}

It does not catch exceptions inside the loop, so if any of these fail -- such as the Connection refused message in this case -- then the map of known facets ends up being empty, and any requests which include a parameter such as project for filtering by facet get a 400.

A suggested improvement (although I don't know how to code myself this as I am not a java programmer) would be to catch exceptions for each shard separately, to build up a facet list based on the ones that respond while ignoring any that generate an exception. If none of the shards are responding, so that despite this the facet map is still empty after calling queryFacets, then maybe it should respond with a 500-series status code, but not a 400.