Skip to content

Commit

Permalink
[Security Solution] [Bug] Index Values are not available in dropdown …
Browse files Browse the repository at this point in the history 
…under New Index Enter for Knowledge Base. (#199610) (#199990)

## Summary

This PR fixes the issue described here
#199610 (comment)

In short, we do not show all indices with `semantic_text` fields. We
only show those which have `semantic_text` fields named `content`.

### Testing notes

Add the index with the `semantic_text` field by running this command in
dev tools:

```
PUT test_index
{
  "mappings": {
  "properties": {
    "attachment": {
      "properties": {
        "content": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "content_length": {
          "type": "long"
        },
        "content_type": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "format": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "language": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    },
    "content2": {
      "type": "semantic_text",
      "inference_id": "elastic-security-ai-assistant-elser2"
    }
    }
}
}
```

You should see the `test_index` index in the `Knowledge Base > New >
Index > Index (dropdown)`.

### Checklist

Delete any items that are not applicable to this PR.

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Steph Milovic <[email protected]>
  • Loading branch information
@e40pud @stephmilovic
e40pud and stephmilovic authored Nov 18, 2024
1 parent 1e29cf8 commit 5ab39e5
Show file tree
Hide file tree
Showing 2 changed files with 68 additions and 14 deletions.
74 changes: 64 additions & 10 deletions ...plugins/elastic_assistant/server/routes/knowledge_base/get_knowledge_base_indices.test.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,15 @@ import { getGetKnowledgeBaseIndicesRequest } from '../../__mocks__/request';

const mockFieldCaps = {
indices: [
'.ds-logs-endpoint.alerts-default-2024.10.31-000001',
'.ds-metrics-endpoint.metadata-default-2024.10.31-000001',
'.internal.alerts-security.alerts-default-000001',
'.ds-.items-default-2024.11.12-000001',
'.ds-.lists-default-2024.11.12-000001',
'.ds-logs-endpoint.alerts-default-2024.11.12-000001',
'.ds-logs-endpoint.events.process-default-2024.11.12-000001',
'gtr-1',
'gtr-with-bug',
'gtr-with-semantic-1',
'metrics-endpoint.metadata_current_default',
'semantic-index-1',
'semantic-index-2',
'search-elastic-security-docs',
],
fields: {
content: {
Expand All @@ -27,9 +30,12 @@ const mockFieldCaps = {
searchable: false,
aggregatable: false,
indices: [
'.ds-logs-endpoint.alerts-default-2024.10.31-000001',
'.ds-metrics-endpoint.metadata-default-2024.10.31-000001',
'.internal.alerts-security.alerts-default-000001',
'.ds-.items-default-2024.11.12-000001',
'.ds-.lists-default-2024.11.12-000001',
'.ds-logs-endpoint.alerts-default-2024.11.12-000001',
'.ds-logs-endpoint.events.process-default-2024.11.12-000001',
'gtr-1',
'gtr-with-bug',
'metrics-endpoint.metadata_current_default',
],
},
Expand All @@ -38,7 +44,55 @@ const mockFieldCaps = {
metadata_field: false,
searchable: true,
aggregatable: false,
indices: ['semantic-index-1', 'semantic-index-2'],
indices: ['gtr-with-semantic-1'],
},
},
ai_embeddings: {
unmapped: {
type: 'unmapped',
metadata_field: false,
searchable: false,
aggregatable: false,
indices: [
'.ds-.items-default-2024.11.12-000001',
'.ds-.lists-default-2024.11.12-000001',
'.ds-logs-endpoint.alerts-default-2024.11.12-000001',
'.ds-logs-endpoint.events.process-default-2024.11.12-000001',
'gtr-1',
'gtr-with-semantic-1',
'metrics-endpoint.metadata_current_default',
],
},
semantic_text: {
type: 'semantic_text',
metadata_field: false,
searchable: true,
aggregatable: false,
indices: ['gtr-with-bug', 'search-elastic-security-docs'],
},
},
semantic_text: {
unmapped: {
type: 'unmapped',
metadata_field: false,
searchable: false,
aggregatable: false,
indices: [
'.ds-.items-default-2024.11.12-000001',
'.ds-.lists-default-2024.11.12-000001',
'.ds-logs-endpoint.alerts-default-2024.11.12-000001',
'.ds-logs-endpoint.events.process-default-2024.11.12-000001',
'gtr-1',
'gtr-with-semantic-1',
'metrics-endpoint.metadata_current_default',
],
},
semantic_text: {
type: 'semantic_text',
metadata_field: false,
searchable: true,
aggregatable: false,
indices: ['search-elastic-security-docs'],
},
},
},
Expand Down Expand Up @@ -66,7 +120,7 @@ describe('Get Knowledge Base Status Route', () => {

expect(response.status).toEqual(200);
expect(response.body).toEqual({
indices: ['semantic-index-1', 'semantic-index-2'],
indices: ['gtr-with-bug', 'gtr-with-semantic-1', 'search-elastic-security-docs'],
});
expect(context.core.elasticsearch.client.asCurrentUser.fieldCaps).toBeCalledWith({
index: '*',
Expand Down
8 changes: 4 additions & 4 deletions x-pack/plugins/elastic_assistant/server/routes/knowledge_base/get_knowledge_base_indices.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ export const getKnowledgeBaseIndicesRoute = (router: ElasticAssistantPluginRoute
include_unmapped: true,
});

const indices = res.fields.content?.semantic_text?.indices;
if (indices) {
body.indices = Array.isArray(indices) ? indices : [indices];
}
body.indices = Object.values(res.fields)
.flatMap((value) => value.semantic_text?.indices ?? [])
.filter((value, index, self) => self.indexOf(value) === index)
.sort();

return response.ok({ body });
} catch (err) {
Expand Down

0 comments on commit 5ab39e5

Please sign in to comment.