Clarisse Integration

[BigRoy]

Issue

This is about an Isotropix Clarisse Integration for Avalon.

Implementation details

• Clarisse supports Python - currently Python 2.X
• You can customize the main menu with init_menus.py and some examples here and here
• There's a special Python Qt helper for Clarisse which requires to use its specialized exec_(). We will need to make sure the Avalon tools run correctly and trigger through this QApplication execution.

# instead of calling app.exec_() like you would do normally in PyQt,
# you call pyqt_clarisse.exec_(app).
pyqt_clarisse.exec_(app)

• And of course there's the Clarisse Python API
• The Clarisse Helper python API seems to have all that's needed for the Work Files API. However for current project name I could find only this old thread but it does seem to be exposed see here and here

# pseudocode
"""Host API required Work Files tool"""
import os

import ix

def file_extensions():
    return [".project"]


def has_unsaved_changes():
    return ix.check_need_save()


def save_file(filepath):
    
    ix.save_project(filepath)

    return filepath


def open_file(filepath):

    ix.open_project(filepath)

    return filepath


def current_file():
    return ix.application.get_current_project_filename()


def work_root():
    from avalon import Session

    work_dir = Session["AVALON_WORKDIR"]
    scene_dir = Session.get("AVALON_SCENEDIR")
    if scene_dir:
        return os.path.join(work_dir, scene_dir)
    else:
        return work_dir

[BigRoy]

@ddesmond has been playing with Avalon and a prototype Clarisse integration and got the basis working - however the Clarisse Qt helper workaround is still the tricky one as it means we'll need to customize how the QApplication instance is created, since you'd need to run exec_() of the QApplication with the special pyqt_clarisse.exec_(app).

It says Qt needs to be imported before the helper is imported, so:

# Allow Clarisse to install the Qt Helper for QApplication
# See: https://www.clarissewiki.com/4.0/sdk/using_pyqt.html
# First we need to import Qt, then import pyqt_clarisse
from avalon.vendor.Qt import QtWidgets
import pyqt_clarisse

This could be in avalon.clarisse.install() so it always happens for Clarisse as host. However, the QtWidgets.QApplication should be executed with pyqt_clarisse.exec_(app) as opposed to app.exec_(). Which means we'd need to somehow influence the behavior defined here.

Any ideas on how to get this operational without hardcoding it in avalon.tools.lib? Ideas @davidlatwe @mottosso ?

I mentioned this potential hack to @ddesmond:

from avalon.vendor.Qt import QtWidgets
import pyqt_clarisse

# Initialize Qt application
app = QtWidgets.QApplication([])
pyqt_clarisse.exec_(app)

# Now because a Qt Application instance exists
# this means the Workfiles tool should be using that one
import avalon.tools.workfiles
avalon.tools.workfiles.show()

Which showed the tool fine since it hits this line in avalon.tools.lib printing:

Using existing QApplication..

However, I still feel it's quite the hack and not how the pyqt_clarisse.exec_(app) helper was intended.
@ddesmond also reported a crash on closing the Qt interface. This is the corresponding crash log:
clarisse.txt

[BigRoy]

@ddesmond You asked me some questions regarding how to potentially implement the ls() and containerise() type of methods for Clarisse as host. I quickly downloaded a Clarisse trial. Here's what I got as a prototype:

import ix
from collections import OrderedDict

# TODO Import this from avalon.pipeline
# from ..pipeline import AVALON_CONTAINER_ID
AVALON_CONTAINER_ID = "pyblish.avalon.container"


def imprint(node, data):
    """Store attributes with value on a node

    Args:
        node (framework.PyOfObject): The node to imprint data on.
        data (dict): Key value pairs of attributes to create.

    Returns:
        None

    """
    for attr, value in data.items():

        # Create the attribute
        node.add_attribute(attr,
                           ix.api.OfAttr.TYPE_STRING,
                           ix.api.OfAttr.CONTAINER_SINGLE,
                           ix.api.OfAttr.VISUAL_HINT_DEFAULT,
                           "avalon")

        # Set the attribute's value
        setattr(node.attrs, attr, value)


def containerise(node, name, namespace, context, loader):
    """Imprint `node` with container metadata.

    Arguments:
        node (framework.PyOfObject: The node to containerise.
        name (str): Name of resulting assembly
        namespace (str): Namespace under which to host container
        context (dict): Asset information
        loader (str): Name of loader used to produce this container.

    Returns:
        None

    """


    data = [
        ("schema", "avalon-core:container-2.0"),
        ("id", AVALON_CONTAINER_ID),
        ("name", name),
        ("namespace", namespace),
        ("loader", str(loader)),
        ("representation", context["representation"]["_id"])
    ]
    # We use an OrderedDict to make sure the attributes
    # are always created in the same order. This is solely
    # to make debugging easier when reading the values in
    # the attribute editor.
    imprint(node, OrderedDict(data))


def parse_container(node):
    """Return the container node's full container data.

    Args:
        node (framework.PyOfObject: A node to parse as container.

    Returns:
        dict: The container schema data for this container node.

    """

    # If not all required data return None
    required = ['id', 'schema', 'name',
                'namespace', 'loader', 'representation']
    if not all(node.attribute_exists(attr) for attr in required):
        return

    data = {attr: getattr(node.attrs, attr)[0] for attr in required}

    # Store the node's name
    data["objectName"] = node.get_full_name()

    # Store reference to the node object
    data["node"] = node

    return data


def ls():    
    """Yields containers from active Clarisse project
    
    This is the host-equivalent of api.ls(), but instead of listing
    assets on disk, it lists assets already loaded in Clarisse; once 
    loaded they are called 'containers'
    
    Yields:
        dict: container
    
    """
    # Iterate all objects in the scene
    # and parse them to see if they are containerised

    # TODO: Allow this to iterate *ALL* objects
    # NOTE: These types are somewhat randomly chosen.
    types = ["GeometryBundleAlembic",
             "GeometryBundleUsd",
             "TextureMapFile",
             "TextureStreamedMapFile",
             "TextureOslFile",
             "LayerFile"]

    nodes = ix.api.OfObjectArray()
    for t in types:
        ix.application.get_factory().get_all_objects(t, nodes)
        for i in range(nodes.get_count()):

            container = parse_container(nodes[i])
            if container:
                yield container

See the TODO in ls() as I'm now filtering to just some random types because I couldn't quickly get it to work to get all types and have it actually return the GeometryBundleAlembic objects. I tried this but it failed:

nodes = ix.api.OfObjectArray()
ix.application.get_factory().get_all_objects(nodes)
for i in range(nodes.get_count()):
    print nodes[i]

# project://__system_vars
# project://__builtin_vars
# project://__custom_vars
# project://__app_prefs_vars
# project://__project_prefs_vars
# project://clone_stamp_3d
# project://particle_paint
# project://property_paint
# project://pick_fit
# project://picker

Hope it helps!

Be aware that I've never used Clarisse before. It's the first time I opened it, so I have no clue whether "containerising" per node makes sense or whether you'd prefer to have it grouped like an actual containered reference or alike. Totally depends on how you end up using loaded content in the app and what you're expecting to load.

The containerise name is somewhat confusing when it actually operates on a single node. That behavior however is similar to Fusion ls() and imprint_container. Usage in a Loader can be seen in this config. Unlike Maya's containerise it doesn actually "contain" multiple objects, that's where the terminology "containerise" originated from. It's not required to have it named containerise at all as explained here so it could be worth refactoring to imprint_container like was done for Fusion.

[mottosso]

I mentioned this potential hack to @ddesmond:

Seems fine to me. Might want to check whether there's already a QApplication running first (they are singletons), but other than that it's just initialisation, which we already do in the host integration anyway (e.g. to install menus).

[BigRoy]

@ddesmond showed me an example of how he usually works with clarisse. Based on that I did some research on how to continue with a first Loader and some additional notes.

Example reference loader (like File > Reference File)

It seems you cannot create custom attributes to "Reference" nodes through the User Interface nor do they have node.attrs exposed in Python API. This is because loaded "file references" are special contexts and are basically similar to "contexts" (folders) and are of OfContext type as opposed to OfObject type.

Note that OfContext does have add_attribute method exposed so it seemed it's possible to still create custom attributes through the API. When I tried to do so with ix.cmds.CreateCustomAttribute however I got this error:

Item 'project://scene/test' is not an editable object.

Nevertheless the actual API does work. So we'll need to make sure to use that workflow.

# Select a reference node and run this
node = ix.selection[0]
node.add_attribute("id",
                   ix.api.OfAttr.TYPE_STRING,
                   ix.api.OfAttr.CONTAINER_SINGLE,
                   ix.api.OfAttr.VISUAL_HINT_DEFAULT,
                   "Avalon")
                   
# Set the value (attributes in Clarisse API are always arrays
# even when it's a single value container) so we use [0]
attr = node.get_attribute("id")
attr[0] = "my custom value"

# Get the value
attr = node.get_attribute("id")
print(attr[0])

Psuedocode Loader example

from avalon import api
import ix

class ReferenceLoader(api.Loader):
    """Reference content into Clarisse"""
    
    label = "Reference"
    families = ["*"]
    representations = ["obj", "abc", "usd", "usda"]
    order = 0
    
    icon = "code-fork"
    color = "orange"
    
    def load(self, context, name=None, namespace=None, data=None):
        
        filepath = self.fname
    
        # Create the file reference
        scene_context = "project://scene"
        paths = [filepath]
        reference = ix.cmds.CreateFileReference(scene_context, paths)
        
        # todo: actually imprint it with data
        # Imprint it with some data so ls() can find this
        # particular loaded content and can return it as a
        # valid container
        # imprint_container(reference,)
    
    def update(self, container, representation):
    
        node = container["node"]
        filepath = api.get_representation_path(representation)
        
        ix.cmds.SetReferenceFilename([node.get_full_name()],
                                     filepath)
        
        # todo: do we need to explicitly trigger reload?
        
    def remove(self, container):
        node = container["node"]
        ix.cmds.DeleteItems([node.get_full_name()])

Clarisse Command Batching

This is extra important when running code from the Qt interface:

This also means that each time the running script modifies item attributes a command will be pushed. To avoid filling the command history, it's recommended to create a batch command. For more information please refer to Command History section.

The code to run in Clarisse might need to be forced into a command batch to make sure undo (if allowed) works logically. Otherwise you might end up with every command being a single undo, potentially leaving scene in a broken state.

ix.begin_command_batch("Load")
# Put your code here
ix.end_command_batch()

Or with a context manager:

import contextlib

@contextlib.contextmanager
def command_batch(name):
    try:
        ix.begin_command_batch(name)
        yield
    finally:
        ix.end_command_batch()

with command_batch("load"):
    # Put your code here

Clarisse Survival Kit importing

@ddesmond pointed me to Clarisse Survival Kit (CSK) as to batch loading assets since it provides a lot of nice functionality to prepare the content. That's totally fine. You'd just use whatever functions they expose to load the data. Since it seems the loaded content is packaged up after load into a folder (its own mini-context) I'd pick that folder as the "container node" and imprint custom data there when possible.
For example the Import Asset UI of CSK triggers the import here using the import_controller() in clarisse_survival_kit.app

It would basically mean adapting the example Loader plugin provided above so that it loads the content exactly the way you'd like. However, the Loader usually loads a specific "publish" and not multiple content that also works separately.

Technically it is possible for a Loader to find and load additional published content, e.g. textures if it can find it. It has just never been done in any integration/config I have seen so far.

[BigRoy]

Some progress:

• Set CLARISSE_STARTUP_SCRIPT environment variable to {core}/setup/clarisse/startup.py to auto-initialize the Menu and tools.
• Draft Avalon + Clarisse integration branch
• Example loader implementation in the Polly config, see Reference Loader