Asset Manager usability issues

[schrockn]

Based on conversations with users and watching them use the tool, I'm convinced that we have some asset manager usability issues.

Automatic splitting based on "." into namespaces I believe is highly problematic. There are just too many cases where periods are sensible and logical parts of names (e.g. file extensions) and then this causes crazy behavior in the UI. We should switch to explicit organization only IMO.

We may also want to introduce an additional layer of ontology, so that there is not just one universal domain of asset keys, but many named domains. It is very easy to pollute the asset manager.

I also believe we should rename the feature to asset catalog.

[schrockn]

cc: @prha for when you get back next week :-) and cc: @sryza @mgasner as this was a topic of interest

[dwallace0723]

Automatic splitting based on "." into namespaces I believe is highly problematic. There are just too many cases where periods are sensible and logical parts of names (e.g. file extensions) and then this causes crazy behavior in the UI. We should switch to explicit organization only IMO.

@schrockn major +1 here. We use dot notation very often and the splitting is a major bummer.

[kinghuang]

I think there's more than just . being used for splitting. The following S3 path:

s3://example-lake-dev.transzap.com/sa-m2/spendnlp/oi-ext/staging/2020-10-05/tables/invoice_detail/year=2019/month=8

Gets turned into this key:

s3.example.lake.dev.transzap.com.sa.m2.spendnlp.oi.ext.staging.2020.10.05.tables.invoice_detail.year.2019.month.8

And this navigation trail:

Assets > s3 > example > lake > dev > transzap > com > sa > m2 > spendnlp > oi > ext > staging > 2020 > 10 > 05 > tables > invoice_detail > year > 2019 > month

So, the = and - characters also went into the split.

Discussion #2950 about partitioned assets plays in this as well. The /year=2019/month=8 part are Hive-style partitions in the path. The asset is logically invoice_detail, with this being one partition of it.