That test of yours, the Voight Kampff test, did you ever take it yourself?

tclobjs

Tohil Tcl objects, tclobjs, are a data type Tohil creates in Python. Tclobjs have gained considerable new power. Among them, they now implement the number protocol.

This means tclobjs can be used for number in numeric calculations without needing to pass through int() or float().

Testing of tclobjs for boolean value now provides tcl semantics. 'f', 'F', 'n', 'N', 0, substrings case-insentiviely matching "false" or "no", evaluate as false; 't', 'T', 'y', 'Y', any number of than 0, substrings matching "true" or "yes", evaulate true.

Tclobjs can be used as one or both operands for addition, subtraction, multiplication, division, remainder, divmod, bitwise or, and, xor, left shift, right shift, etc. Unary ops invert, negative, position, absolute value all work.

Tclobjs can be used for "inplace" number calculations such as +=, /=, <<=, etc.

Tclobj iterator code was rewritten from Python into C.

Removed almost all of the as_" methods of tclobjs that convert tclobjs into various Python data types. tclobj.as_int() has been replaced by int(tclobj), as_bool by bool(), as_float by float(), as_str by str(). llength() has been removed; you can use len() to get the same thing.

Removed tclobj's as_tuple() method; use tuple(tclobj) instead. Likewise removed as_tclobj() method; use tohil.tclobi(tclobj) instead. Remove as_tcldict() ; use tohil.tcldict(tclobj) instead.

What's cool is these functions are provided by the tclobj type implementation, so that are real efficient in terms of how they interact with the underlying Tcl objects.

The tclobj.reset() method has been renamed to clear() for consistency with Python lists and dicts. It also works for tcldicts.

Tclobj lappend method has been renamed to append and lappend_list renamed to extend, for compatibility with Python's lists.

• tclobjs can now ingest python sets (in addition to lists, tuples, etc, which it already could do.)

Tclobj Shadow Vars

Another new feature, Tclobj shadow vars, t = tohil.tclvar('t'), makes t a tclobj that shadows a variable t in the Tcl interpreter. Any changes to the variable from the Tcl side are "seen" from the Python side, and vice versa. The variable can also be an array element.

Tcldicts

• Tcldict objects now provide many methods that standard Python dicts provide, such as keys(), values(), items(). Because of this, dict(tcldict) now works.

• The clear() method is now supported to empty the Tcl dict.

• A new tcldict pop method behaves the same as pop for standard Python dicts, popping the last item in the list if no position is specified, else popping the specified position, i.e. removing it from the list and returning it.

• A new .insert(i, x) method will insert item x at position i.

ShadowDicts

ShadowDicts implement many additional methods implemented by standard Python dicts.

ShadowDicts now have a get method that behaves as standard dicts do. A new clear method removes all items from the shadow diction, i.e. it unsets the shadowed Tcl array.

Python exception improvements

We now raise more standard Python TypeError, KeyError and ValueError exceptions in places where we used to just raise RuntimeError.

Tohil method improvements and changes

Tohil.unset can now take an arbitrary number of arguments of variable names and array elements to unset, include zero. As before, it is fine to unset something that doesn't exist.

Testing Improvements

• Lots of new tests.
• Also we're now using the hypothesis testing framework and have found
  and fixed a number of problems because of it.
• All tests pass now on 32-bit ARM Linux.
• Linux CI automated testing using Github Actions

We began to recognize in them, strange obsessions

Minor improvements not worthy of a point release because they didn't change anything operationally meaningful

• String representations of tclobjs and tcldicts returned by repr() and capped around 100 characters.

Bug Fixes

• Use of dlopen() to load the python shared library before initializing python fixes a problem on Linux where if tcl started python (versus python starting from the command line), loads of python modules that include shared libraries (such as "import sqlite3") would fail, because the shared library being loaded would fail to find the python functions they were trying to call.
• tcldicts can now be compared for equality et al as tclobjs can (sort of a minor improvement but can also be seen as a bug fix. Does anyone read these notes, anyway?)

Build Tooling Improvements

• By figuring out the name of the tcl library (libtcl8.6... on Linux and Mac, libtcl86 on FreeBSD), you no longer have to edit the generated setup.py by hand to get tohil to link on FreeBSD.
• If tohil is initializing python, because tcl existed first, try to dig argv[0] out of tcl and if successful, convert to wide and call Py_SetProgramName with it. This is recommended practice for python and is supposed to help steer python to the right stuff when we initialize it. (We hoped this would fix that Linux problem mentioned above, but it didn't.)
• The generated pkgIndex.tcl and setup.py files are no longer removed by make clean, only by make distclean.

Code cleanup, compliance, and a little bit looking toward the future

• a pointer to the tcl interpreter (Tcl_Interp *) is now stored in tcldicts and tclobjs, removing most references to the global tcl interpreter (although many remain)
• Several tohil C functions renamed to be more in compliance with PEP-7.

Fiery the angels fell; deep thunder rolled around their shores; burning with the fires of Orc.

New Features

• New tcldict python type provides strong, pythonic semantics for accessing tcl dictionaries
• Conversion type "to" can be set as an attribute of tclobj and tcldict objects
• td_* access methods removed from tclobj type -- use tcldict instead for more fulent, native behavior of indexing, checking for existence, deleting, length, iterating, etc.
• tclobj and tcldict tcl object type and reference count are now accessed via _refcount and _tcltype attributes.
• tohil.__version__ set with tohil package version

Bug Fixes and Improvements

• Improved UTF-8 support and bug fixes.
• Problem on Linux when starting from tcl, where tcl loaded tohil, tohil brought up python, python loaded the tohil C library, but the tohil library had unresolved symbols calling python, has been fixed, or at least "fixed" (all tests pass).
• Build pickier finding the right version of python3-config to use

A new life awaits you in the Off-world colonies. The chance to begin again in a golden land of opportunity and adventure.

Welcome to Tohil 3!

Tohil 3 brings forward all the slick stuff from Tohil 2, plus it provides the
means of accessing Tcl functions from python in such a way that they very much
look and behave like native python functions. Not only that, but for tcl procs
made available to python by tohil, every parameter can be specified by
position or by name, something few native python functions or python C
functions can do.

TclProcs

Any Tcl proc or C command can be defined as a python function simply
by creating a TclProc object and then calling it.

>>> import tohil
>>> tohil.package_require("Tclx")
'8.6'
>>> intersect = tohil.TclProc("intersect")
>>> intersect([1, 2, 3, 4, 5, 6], [4, 5, 6, 7, 8, 9], to=list)
['4', '5', '6']

It's pretty fun to play with them this way from the command line.

While TclProcs are directly callable, as seen above, they are fully
fledged python object and have a number of interesting and potentially
useful attributes and method, including the python function name,
tcl proc name, whether the tcl function being shadowed is a proc or
not (if not, it's a command written in C), a python dictionary
specifying any default arguments and their values, and the proc's arguments.

TclNamespaces

But wait, there's more. tohil.import_namespace(my_namespace) will create
a TclNamespace object and import all the procs and C commands as methods
of that namespace, recursively importing any subordinate namespaces and
their procs and C commands as well. Namespaces and function calls can
be chained, so you get the hierarchy of tcl namespaces and procs and
C commands created after loading all of your packages, chainable
from python.

It's a convenient way to leverage TclProcs across all of your tcl procs
and commands.

>>> import tohil
>>> tohil.package_require("clock::rfc2822")
'0.1'
>>> tcl = tohil.import_tcl()
>>> tcl.clock.rfc2822.parse_date('Wed, 14 Apr 2021 12:04:48 -0500', to=int)
1618419888

TclError Exception Class

Tohil 3 also adds a sweet TclError exception class, and any tcl errors that
bubble back all the way to python without any tcl code having caught the
error will be thrown in python as TclError exceptions. The TclError object
can be examined to find out all the stuff Tcl knows about the error...
the result, the error code, code level, error stack, traceback, and error line.

New helpers Functions

tohil.package_require is real useful. The others ones tohil needs for itself
and they're not as useful, but maybe for some people for some purposes.

• tohil.package_require(package_name, version=version)
• tohil.info_procs() - return a list of procs. pattern arg optional.
• tohil.info_commands() - return a list of commands, includes procs and C commands.
• tohil.info_body() - return the body of a proc.
• tohil.info_default() - return the default value for an argument of a proc
• tohil.info_args(proc) - return a list of the names of the arguments for a proc
• tohil.namespace_children(namespace) - return a list of all the child namespaces of a namespace

Tests

• Dozens of new tests.
• "make test" runs both the python ones and the tcl ones

Crazy Cool Cat Achieves Orbit

Welcome to Tohil 2.

Tohil 2 brings many improvements, easily worth a bump in the major release number, plus many bug fixes and stability improvements.

Tohil 2 introduces a new Python data type called a tclobj. It's really cool. It gives you a python object that contains a tcl object within it. You can make tclobj objects from just about any python object using t = tohil.tclobj(myPythonObject). Or you can make an empty one with t = tohil.tclobj(). You can fill a tclobj with data from a tcl variable or array element, get its contents as many different python data types, and store its contents back into a tcl variable or array element if you'd like.

Once you've got a tclobj you can covert it to one of many python object types using (assuming your object is named 't') t.as_bool(), t.as_bytearray(), t.as_dict(), t.as_float(), t.as_int(), t.as_list(), t.as_set(), t.as_str(), t.as_tclobj(), and t.as_tuple.

You can set a tclobj to the contents of a tcl variable using t.getvar(), and set the contents of the tclobj into a tcl variable using t.setvar().

You can treat a tclobj object as a list, using t.llength() to see the size of the list, t.lindex(i) to get the i'th element of the list, t.lappend() to lappend to it, and use t.lappend_list() to lappend a list to it. Indeed you can use Python syntax for manipulating lists with tclobjs that are being used as lists. For examaple if t contains a list, t[3] will give you the third element, and saying t[3] = 'bar' will set an element in the list to a value. Slice notation is also supported.

You can reset a tclobj with t.reset(), set it to some arbitrary pyton object with t.set(), and check the tcl object's reference count with t.refcount(), which is kind of insane.

Further, tclobj provides methods in support of tcl dictionaries. t.td_exists() will see if a key exists in the dictionary, t.td_get will get it, t.td_remove will remove it, t.td_set will set it, and t.td_size will tell you its size. A python list of keys can be specified to navigate a hierarchy of dictionaries.

Fixed some crashes where we were causing native cpython data types to be deleted because we were improperly decrementing their reference counts.

tclobjs containing lists can be iterated on with "for i in tclobj" style usage.

And as with get t.td_get() takes a default value and if specified and the key isn't there, you get the default instead. In version 2, if to= is specified, the default value is pushed through that same conversion if a default value and a to= conversion are specified.

tclobj's can be compared to each other.

tohil seems to work pretty well, but it's immature, and there are probably bugs, and almost certainly some memory leaks.

Nonetheless, tohil 2 brings a huge improvement in capabilities and reduction in crashes. I hope you enjoy using it as much as I enjoyed making it.

Karl Lehenbauer / April 4, 2021

Crazy Cool Cat

New Features

• tohil::interact allows you to enter the python interactive command loop from the tcl command line
• tohil.interact allows you to enter the tcl interactive command loop from the python command line
• new tohil::run command execs python code from tcl but snags and returns anything emitted to stdout while the code was executing

Misc Improvements

• 21 new python tests
• setup.py.in -> setup.py autoconf substitution now gets the package version set in configure.ac

Bug Fixes

• tohil::call --kwlist actually works now
• It is no longer a coredump if tohil.call gets a tcl error

Ruggedized Tohil

New capabilities

• You can now specify to=type as an optional argument when creating shadow dictionaries.

Bug fixes

• We no longer try to maintain local state in the shared library. We stash state in the python interpreter instead so that if we aren't running the same shared library between tcl and python, we still work properly. (It's useful to build the python module using python tooling and the tcl module using tcl tooling, even though it's the same C file.) Previously with separate shared libraries, python would create and use a new tcl interpreter, rather than finding the existing one, which is what we want.
• Fix broken exception handling when there's no traceback from the python side because you executed python.exec at the top level
• 32 tests now from the tcl side

Tohil, a feathered serpent, integrates Python, the serpent, and TCL, the feather.

• builds out of the box on Linux and FreeBSD.

From the Tcl side...

• can do python equivalents of eval and exec using tohil::eval and tohil::exec
• provides a "call" function (tohil::call) that can call python where each argument is specified separately, keeping data from being potentially interpreted by tcl, and provides the ability to specify key-value args (kwargs) to python functions that can use them.
• provides a run function that works like exec but grabs anything emitted to python's stdout and returns it to the caller

From the Python side...

• can evaluate code in the Tcl interpreter using tohil.eval
• can call Tcl functions with a tohil.call function that takes explicit arguments, keeping data from being potentially interpreted by python
• can get, set, check the existence of, and unset, variables, array elements, and arrays in the Tcl interpreter.
• can evaluate tcl expressions with tohil.expr, run tcl's subst over texting with tohil.subst
• developer can specify, for most functions, a python data type expected, such as int, float, str, tuple, list, set, dict, etc.
• use shadow dictionaries (tohil.ShadowDict) to get a dict-like object in python that shadows a Tcl array