[WIP] implement vald benchmark #1

kmrmt · 2023-07-18T06:50:50Z

run benchmark

$ pushd ./engine/servers/vald-single-node/ && docker compose up -d && popd
$ python -m run --engines vald-single-node-fp32 --datasets glove-25-angular

sonatype-lift · 2023-07-18T06:50:53Z

Sonatype Lift is retiring

Sonatype Lift will be retiring on Sep 12, 2023, with its analysis stopping on Aug 12, 2023. We understand that this news may come as a disappointment, and Sonatype is committed to helping you transition off it seamlessly. If you’d like to retain your data, please export your issues from the web console.
We are extremely grateful and thank you for your support over the years.

📖 Read about the impacts and timeline

sonatype-lift · 2023-07-18T06:53:09Z

engine/clients/vald/__init__.py

@@ -0,0 +1,3 @@
+from engine.clients.vald.configure import ValdConfigurator


F401: 'engine.clients.vald.configure.ValdConfigurator' imported but unused

❗❗ 3 similar findings have been found in this PR

🔎 Expand here to view all instances of this finding

File Path Line Number

engine/clients/vald/init.py 2

engine/clients/vald/init.py 3

engine/clients/vald/parser.py 1

Visit the Lift Web Console to find more details in your report.

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:11Z

engine/clients/vald/__init__.py

@@ -0,0 +1,3 @@
+from engine.clients.vald.configure import ValdConfigurator
+from engine.clients.vald.search import ValdSearcher
+from engine.clients.vald.upload import ValdUploader


W292: no newline at end of file

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:12Z

engine/clients/vald/config.py

+    )
+
+
+_label_selector = client.V1LabelSelector(match_labels={"app": "vald-agent-ngt"})


E501: line too long (80 > 79 characters)

❗❗ 15 similar findings have been found in this PR

🔎 Expand here to view all instances of this finding

File Path Line Number

engine/clients/vald/config.py 24

engine/clients/vald/config.py 38

engine/clients/vald/config.py 82

engine/clients/vald/config.py 114

engine/clients/vald/config.py 124

engine/clients/vald/config.py 133

engine/clients/vald/config.py 200

engine/clients/vald/configure.py 17

engine/clients/vald/parser.py 7

engine/clients/vald/parser.py 10

Showing 10 of 15 findings. Visit the Lift Web Console to see all.

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:14Z

engine/clients/vald/configure.py

+        )
+        if len(res.items) > 0:
+            scheduling_api.delete_priority_class(PRIORITY_CLASS.metadata.name)
+


W293: blank line contains whitespace

❗❗ 3 similar findings have been found in this PR

🔎 Expand here to view all instances of this finding

File Path Line Number

engine/clients/vald/parser.py 9

engine/clients/vald/parser.py 12

engine/clients/vald/parser.py 15

Visit the Lift Web Console to find more details in your report.

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:16Z

engine/clients/vald/configure.py

+        if len(res.items) > 0:
+            scheduling_api.delete_priority_class(PRIORITY_CLASS.metadata.name)
+
+        time.sleep(10) # TODO: using watch


E261: at least two spaces before inline comment

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:18Z

engine/clients/vald/configure.py

+        scheduling_api = k8s.client.SchedulingV1Api(api_client)
+        scheduling_api.create_priority_class(PRIORITY_CLASS)
+
+        time.sleep(30) # TODO: using watch


E261: at least two spaces before inline comment

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:19Z

engine/clients/vald/config.py

@@ -0,0 +1,237 @@
+from kubernetes import client


E0401: Unable to import 'kubernetes'

❗❗ 8 similar findings have been found in this PR

🔎 Expand here to view all instances of this finding

File Path Line Number

engine/clients/vald/configure.py 1

engine/clients/vald/search.py 2

engine/clients/vald/search.py 4

engine/clients/vald/search.py 5

engine/clients/vald/upload.py 1

engine/clients/vald/upload.py 5

engine/clients/vald/upload.py 6

engine/clients/vald/upload.py 7

Visit the Lift Web Console to find more details in your report.

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:21Z

engine/clients/vald/config.py

@@ -0,0 +1,237 @@
+from kubernetes import client


reportMissingImports: Import "kubernetes" could not be resolved

❗❗ 8 similar findings have been found in this PR

🔎 Expand here to view all instances of this finding

File Path Line Number

engine/clients/vald/configure.py 1

engine/clients/vald/search.py 2

engine/clients/vald/search.py 4

engine/clients/vald/search.py 5

engine/clients/vald/upload.py 1

engine/clients/vald/upload.py 5

engine/clients/vald/upload.py 6

engine/clients/vald/upload.py 7

Visit the Lift Web Console to find more details in your report.

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:23Z

engine/clients/vald/configure.py

+                        **collection_params["ngt_config"],
+                        **{
+                            "dimension": dataset.config.vector_size,
+                            "distance_type": self.DISTANCE_MAPPING[


reportGeneralTypeIssues: Argument of type "str" cannot be assigned to parameter "__key" of type "Distance" in function "getitem"
"str" is incompatible with "Distance"

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

sonatype-lift · 2023-07-18T06:53:25Z

engine/clients/vald/search.py

+
+
+class ValdSearcher(BaseSearcher):
+    cfg: Dict = None


reportGeneralTypeIssues: Expression of type "None" cannot be assigned to declared type "Dict[Unknown, Unknown]"
Type "None" cannot be assigned to type "Dict[Unknown, Unknown]"

ℹ️ Expand to see all @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

implement vald benchmark

7b2e8c2

sonatype-lift bot reviewed Jul 18, 2023

View reviewed changes

kmrmt changed the title ~~implement vald benchmark~~ [WIP] implement vald benchmark Jul 19, 2023

kmrmt added 4 commits July 19, 2023 20:41

fix

73e9dc2

fix

5ba35e8

fix

1976038

fix

116b0b6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] implement vald benchmark #1

[WIP] implement vald benchmark #1

kmrmt commented Jul 18, 2023 •

edited

Loading

sonatype-lift bot commented Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

sonatype-lift bot Jul 18, 2023

		@@ -0,0 +1,3 @@
		from engine.clients.vald.configure import ValdConfigurator

File Path	Line Number
engine/clients/vald/init.py	2
engine/clients/vald/init.py	3
engine/clients/vald/parser.py	1

Command	Usage
`@sonatype-lift ignore`	Leave out the above finding from this PR
`@sonatype-lift ignoreall`	Leave out all the existing findings from this PR
`@sonatype-lift exclude <file\|issue\|path\|tool>`	Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file

		)


		_label_selector = client.V1LabelSelector(match_labels={"app": "vald-agent-ngt"})

[WIP] implement vald benchmark #1

Are you sure you want to change the base?

[WIP] implement vald benchmark #1

Conversation

kmrmt commented Jul 18, 2023 • edited Loading

sonatype-lift bot commented Jul 18, 2023

Sonatype Lift is retiring

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jul 18, 2023

Choose a reason for hiding this comment

kmrmt commented Jul 18, 2023 •

edited

Loading