Python libraries and scripts to talk to the NPO api’s. Currently,classes are available for the frontend api (Media, Pages, Schedule, and Screens) and for the media backend api (MediaBackend, Pages Backend and Thesaurus).
There are several ways to use this.
-
You can install it directly in your python3 environment or, recommendedly, install it in a dedicated virtualenv. If you want it as a library, this is probably the way.
-
The provided command line utilities can also be called from a ready-made docker container. It requires docker and a few hundred Mb’s but for the rest this should be simple.
Then you can install the python libraries and command line scripts with the following command
pip3 install git+https://github.com/npo-poms/[email protected]
or if you dare install the latest, which may sometimes be a bit broken:
pip3 install --upgrade 'git+https://github.com/npo-poms/pyapi.git'
Needed dependencies will be installed automatically, and are maintained in pyproject.toml.
It may also be a good idea to set up a virtual environment instead, to avoid version conflicts with other python projects.
E.g. like so:
mihxil@baleno:~$ python3 -m venv ~/venvs/pyapi-env
mihxil@baleno:~$ source ~/venvs/pyapi-env/bin/activate
(pyapi-env) mihxil@baleno:~$ pip3 install --upgrade 'git+https://github.com/npo-poms/pyapi.git'
(pyapi-env) mihxil@baleno:~$ which npo_media_get
/Users/mihxil/venvs/pyapi-env/bin/npo_media_getTo run tests also its dependencies must be installed
pip install -e .[testing]If you don’t feel like installing anything at all, but have a docker environment running (amd64), then I’ve published a docker image for you:
docker run -i -v ~/conf:/root/conf mihxil/npo-pyapi npo_mediabackend_get -c POMS_VPRO_729992Nothing more needed then.
You can also e.g. define an alias to help you remember
alias npoapi='docker run -e ENV=`eval ''echo $ENV'a'` -v ~/conf:/root/conf -v $(pwd):/work mihxil/npo-pyapi'And then all tools are available like so:
# frontend
npoapi media_get --help
npoapi media_changes --help
npoapi media_follow_changes --help
npoapi media_iterate --help
npoapi media_search --help
npoapi pages_get --help
npoapi pages_search --help
npoapi schedule_get --help
npoapi schedule_search --help
npoapi check_credentials --help
#backend
npoapi mediabackend --help
npoapi mediabackend_get --help
npoapi pagesbackend --help(The 'npo_' prefixes are made optional in this docker image, and left away here)
Just to use it as a python on the current directory
docker run -it -v ~/conf:/root/conf:z -v $(pwd):/work:z mihxil/npo-pyapi /bin/shThis can be useful to run scripts only using the API.
Check pyproject.toml to see which command line clients it will make available. These also serve as examples on how to use the npoapi module.
E.g. you can do:
michiel@belono:~$ npo_media_get --createconfig POMS_NTR_388772
No configuration file found. Now creating.
Your NPO api key?:
Your NPO api secret?:
Your NPO api origin?:{"objectType":"program","mid":"POMS_NTR_388772","type":"BROADCAST","avType":"AUDIO","workflow":"PUBLISHED","sortDate":1376395200000,"creationDate":1376435075424,"lastModified":1376435112166,"urn":"urn:vpro:media:program:28506247","embeddable":true,"episodeOf":[{"midRef":"AUTO_WINFRIEDDRAAITDOOR","urnRef":"urn:vpro:media:group:13405810","type":"SERIES","index":1,"highlighted":false,"added":1376435078278}],"crids":["crid://broadcast.radiobox2/203820"],"broadcasters":[{"id":"NTR","value":"NTR"}],"titles":[{"value":"Winfried Draait Door","owner":"RADIOBOX","type":"MAIN"}],"descriptions":[{"value":"Elke werkdag draait Winfried Baijens door op Radio 6 met de beste soul en jazz, nieuwe releases, Nederlands talent en de mooiste prijzen. Geen dag gaat voorbij zonder een thema dat veelal iets te maken heeft met de actualiteit. Voorwaarde is; het thema moet allitereren. Daar houdt Winfried namelijk van, allitereren.\nVerder hoor je berichten van nationale en internationale sterren, luisteraars, betrokkenen bij het thema en muziekvrienden die Winfrieds voicemail inspreken. DJ Git Hyper is een vaste gast en Winfried maakt ook een muzikale kettingbrief. Vele grote namen uit de Nederlandse muziekwereld werkten al mee aan deze multitracks.","owner":"RADIOBOX","type":"MAIN"}],"genres":[],"countries":[],"languages":[],"duration":7200000,"descendantOf":[{"midRef":"AUTO_WINFRIEDDRAAITDOOR","urnRef":"urn:vpro:media:group:13405810","type":"SERIES"},{"midRef":"POMS_S_VPRO_171668","urnRef":"urn:vpro:media:group:14683553","type":"ARCHIVE"},{"midRef":"POMS_S_VPRO_218686","urnRef":"urn:vpro:media:group:14921825","type":"ARCHIVE"},{"midRef":"POMS_S_VPRO_117474","urnRef":"urn:vpro:media:group:20347947","type":"PLAYLIST"}],"email":["[email protected]"],"websites":[{"value":"http://www.radio6.nl/winfrieddraaitdoor"}],"predictions":[{"state":"REALIZED","platform":"INTERNETVOD"}],"locations":[{"programUrl":"http://download.omroep.nl/audiologging/r6/2013/08/13/1400_1600_winfried_draait_door.mp3","avAttributes":{"avFileFormat":"MP3"},"duration":7200000,"owner":"RADIOBOX","creationDate":1376435052113,"lastModified":1376435075571,"workflow":"PUBLISHED","urn":"urn:vpro:media:location:28506251"}],"scheduleEvents":[{"start":1376395200000,"duration":7200000,"poProgID":"POMS_NTR_388772","channel":"RAD6","urnRef":"urn:vpro:media:program:28506247","midRef":"POMS_NTR_388772"}],"images":[{"title":"winfried_baijens.jpg","description":"Winfried Draait Door","imageUri":"urn:vpro:image:121034","owner":"RADIOBOX","type":"PICTURE","highlighted":false,"creationDate":1376435059364,"lastModified":1376435075570,"workflow":"PUBLISHED","urn":"urn:vpro:media:image:28506249"}]}Or e.g.
michiel@baleno:~$ ENV=test npo_schedule_get | jq{
"total": 850553,
"offset": 0,
"max": 10,
"items": [
{
"channel": "NED1",
"start": 74016000000,
"guideDay": 73954800000,
"duration": 2062000,
"urnRef": "urn:vpro:media:program:19222530",
"midRef": "immix_116032",
"media": {
"objectType": "program",
"mid": "immix_116032",
"type": "BROADCAST",
"avType": "VIDEO",
"sortDate": 74016000000,
...More information about command line options can be gotten with '-h'
michiel@belono:~$ npo_media_get -h
usage: npo_media_get.py [-h] [-s {asc,desc}] [-a {json,xml}]
[-e {prod,acc,test}] [-d]
mid [{descendants,members,episodes,related,}]
Get an media object from the NPO Frontend API
positional arguments:
mid The mid of the object to get
{descendants,members,episodes,related,}
Sub call for the mediaobject. On default the
mediaobject itself is returned, but ou can also opt
for one of these choices
optional arguments:
-h, --help show this help message and exit
-s {asc,desc}, --sort {asc,desc}
sort (only relevant when using sub)
-a {json,xml}, --accept {json,xml}
-e {prod,acc,test}, --env {prod,acc,test}
-d, --debug
DEBUG=true and ENV=<test|acc|prod> environment variables are recognized.
Credentials are read from a config file. If such a file does not exist it will
offer to create one.The 'npo_mediabackend_get' call supports a –process options, this works like so:
michiel@belono:~$ npo_mediabackend_get -e prod POMS_S_VPRO_3512033 --process "update.duration='PT5M'"<?xml version="1.0" ?>
<group avType="MIXED" embeddable="true" mid="POMS_S_VPRO_3512033" ordered="true" type="PLAYLIST" urn="urn:vpro:media:group:72865615" xmlns="urn:vpro:media:update:2009">
<broadcaster>VPRO</broadcaster>
<broadcaster>NTR</broadcaster>
<portal>NETINNL</portal>
<title type="MAIN">NetInNl</title>
<duration>PT5M</duration>
<locations/>
<scheduleEvents/>
<images/>
<poSeriesID>POMS_S_VPRO_3512033</poSeriesID>
</group>This way a poms object can be edited using python. The resulting XML can be posted back.
The incoming object is an unmarshalled python object. Originally, can currently still the defaul this is done by PyXB
Because POMS provides XSD schemas for all objects it can return and receive, it is feasible to make object bindings automatically (in java that would e.g. be done by jaxb)
Originally this was done with the classes (generated by pyxb) in the npoapi.xml module. These classes depend on pyxb itself, which reached end of life in 2018, and it can be expected that in newer python versions this will no longer work.
Support for xsdata was added as an alternative. Binding can be found in the `npoapi.data module. These generated classes are plain dataclasses, but with support to unmarshall from XML and marshall to XML.
Some relevant methods now have a 'bindings' parameter to switch between implementations.
The pyxb version is now deprecated and will be dropped as soon as an alternative is finished and tested well enough. The 'bindings' parameter will then be removed then too.
POMS API’s normally support both XML and JSON. But in some cases (backend api’s) only XML, and some other cases (changes feeds) only JSON is supported.
|
Note
|
The Backend API of POMS has more and more support for json now too. |
A generic binding to and from JSON would probably require manual tweaking. The poms java domain objects are annotated with JAXB and jackson annotations to arrange json bindings. The information and customizations contained in the jackson annotations are not available in the XML schema’s, or currently available in another way (besides the java code itself).
Json fortunately quite naturally binds to schemaless python structures, so I think the advice would be to just be schemaless if you want to use JSON.
Credentials and other setting for the different api’s can be manually added and maintained in a file USER_HOME/conf/creds.properties It looks for example like this
# npo api
apikey=<your key>
secret=<your secret>
origin=http://www.vpro.nl
# backend api
user=vpro-mediatools:<your password>
user.prod=vpro-mediatools:<your password or prod>
email[email protected]Command line clients offer the --createconfig option to create this file for you if it doesn’t exist.
Tests can be run like so:
python3 -m unittest discover -s tests -p '*_test.py'or like so if nosetests is installed:
nosetestsThe libraries and scripts in this repository are all completely generic. In https://github.com/npo-poms/scripts we collect more specific scripts, to perform certain tasks like 'link an image to all members of a group', or 'check the consistency of the pages api'.