reduction_factor -> bottleneck_size

[calpt]

closes adapter-hub/website#25

[JoPfeiff]

looks good, but requires manual changes in nothing_to_see_here? First add a PR there and then we merge simultaneoulsy?

[calpt]

looks good, but requires manual changes in nothing_to_see_here? First add a PR there and then we merge simultaneoulsy?

We would require a manual change of all currently uploaded adapters (info card + zip folder on the server)

[JoPfeiff]

Yeah thats what I thought.
Alternatively, how much effort does it involve to have both settings available?
A user is required to set either? In the frontend we would present the bottleneck size.

[calpt]

That would certainly be possible for the yaml files. Not sure how much overhead it causes on the library side as we would have to process configs with different keys but same semantics (could be a problem for hashing, resolving etc).

[JoPfeiff]

makes sense.
To bypass that we could always translate the reduction_factor into bottleneck_size.
This is always possible, however not the other way around:

hidden_size = 768
reduction_factor = 32 => bottleneck_size = 24

however

hidden_size = 768
bottleneck_size = 100 => reduction_factor = 7.68 

we would not want to store floats here...

We could thus only hash the bottleneck_size?

[calpt]

yes, I think something like this would be possible.

just for clarification: we now always assume a hidden size of 768, right? So the conversion would be reduction_factor = 768 / bottleneck_size regardless of the actual model used (compared to previously, where the reduction factor was dependent on the actual hidden size).

[JoPfeiff]

No, the reduction_factor is always and remains dependent on the actual hidden_size.
So for Base the hidden_size = 768, for Large its hidden_size = 1024 (I think).
If reduction_factor is set, the bottleneck_size should be calculated according to the actual hidden_size

[JoPfeiff]

this has not been implemented yet correct?

[calpt]

Not yet, I'll try to do it as soon as possible (but probably not before the weekend 😅)

[JoPfeiff]

No problem at all, I just had the PR set for approved and was worried that someone will merge it :)

[calpt]

@JoPfeiff how important is the implementation of this change at the moment? Imho a "clean" solution supporting both options is a bit tricky.

[JoPfeiff]

not a priority. we can get back to it later