A command line interface and Python module for accessing the CKAN Action API
tested under Python 2.6, 2.7, 3.2, 3.3 and pypy
The ckanapi command line interface lets you access local and remote CKAN instances for bulk operations and simple API actions.
Simple actions with string parameters may be called directly. The response is pretty-printed to STDOUT. e.g.:
$ ckanapi action group_list -r http://demo.ckan.org
[
"data-expolorer",
"example-group",
"geo-examples",
...
]
Local CKAN actions may be run by specifying the config file with -c. If no remote server or config file is specified the CLI will look for a development.ini file in the current directory, much like paster commands. When connecting to a local CKAN instance the site user (sysadmin) is used by default.
Datasets, groups and organizations may be dumped to JSON lines text files and created or updated from JSON lines text files. These bulk dumping and loading jobs can be run in parallel with multiple worker processes. The jobs in progress, the rate of job completion and any individual errors are shown on STDERR while the jobs run.
e.g. load datasets from a dataset dump file with 3 processes in parallel:
$ ckanapi load datasets -I datasets.jsonl.gz -z -p 3 -c /etc/ckan/production.ini
Bulk loading jobs may be resumed from the last completed record or split across multiple servers by specifying record start and max values.
Datasets may be exported to the datapackage.json format (which includes the actual resources, where available).
If the resource url is not available, the resource will be included in the datapackage.json file but the actual resource data will not be downloaded.
$ ckanapi dump datasets --all --datapackages=./output_directory/ -r http://sourceckan.example.com
Simple shell pipelines are possible with the CLI. E.g. update the title of a dataset with the help of the 'jq' command-line json tool:
$ ckanapi action package_show id=my-dataset \
| jq '.+{"title":"New title"}' \
| ckanapi action package_update -i
E.g. Copy all datasets from one CKAN instance to another:
$ ckanapi dump datasets --all -q -r http://sourceckan.example.com \
| ckanapi load datasets
The ckanapi Python module may be used from within a CKAN extension or in a Python 2 or Python 3 application separate from CKAN.
Making a request:
import ckanapi
demo = ckanapi.RemoteCKAN('http://demo.ckan.org',
user_agent='ckanapiexample/1.0 (+http://example.com/my/website)')
groups = demo.action.group_list(id='data-explorer')
print groups
result:
[u'data-explorer', u'example-group', u'geo-examples', u'skeenawild']
The example above is using an "action shortcut". The .action
object detects
the method name used ("group_list" above) and converts it to a normal
call_action
call. This is equivalent code without using an action shortcut:
groups = demo.call_action('group_list', {'id': 'data-explorer'})
All actions in the CKAN Action API
and actions added by CKAN plugins are supported by action shortcuts and
call_action
calls.
NotAuthorized
- user unauthorized or accessing a deleted itemNotFound
- name/id not foundValidationError
- field errors listed in.error_dict
SearchQueryError
- error reported from SOLR indexSearchError
CKANAPIError
- incorrect use of ckanapi or unable to parse responseServerIncompatibleError
- the remote API is not a CKAN API
When using an action shortcut or the call_action
method
failures are raised as exceptions just like when calling get_action
from a
CKAN plugin:
import ckanapi
demo = ckanapi.RemoteCKAN('http://demo.ckan.org',
apikey='phony-key',
user_agent='ckanapiexample/1.0 (+http://example.com/my/website)')
try:
pkg = demo.action.package_create(name='my-dataset', title='not going to work')
except ckanapi.NotAuthorized:
print 'denied'
When it is possible to import ckan
all the ckanapi exception classes are
replaced with the CKAN exceptions with the same names.
File uploads for CKAN 2.2+ are supported by passing file-like objects to action shortcut methods:
import ckanapi
mysite = ckanapi.RemoteCKAN('http://myckan.example.com',
apikey='real-key',
user_agent='ckanapiexample/1.0 (+http://example.com/my/website)')
mysite.action.resource_create(
package_id='my-dataset-with-files',
upload=open('/path/to/file/to/upload.csv'))
When using call_action
you must pass file objects separately:
mysite.call_action('resource_create',
{'package_id': 'my-dataset-with-files'},
files={'upload': open('/path/to/file/to/upload.csv')})
A similar class is provided for accessing local CKAN instances from a plugin in the same way as remote CKAN instances. This class defaults to using the site user with full access.
import ckanapi
registry = ckanapi.LocalCKAN()
try:
registry.action.package_create(name='my-dataset', title='this will work fine')
except ckanapi.ValidationError:
print 'unless my-dataset already exists'
A class is provided for making action requests to a webtest.TestApp instance for use in CKAN tests:
import ckanapi
import webtest
test_app = webtest.TestApp(...)
demo = ckanapi.TestAppCKAN(test_app, apikey='my-test-key')
groups = demo.action.group_list(id='data-explorer')