jsnapy as a python module: how to retrieve the ERROR: and FAIL | messages

[nskalis]

Hi,

I would like to use jsnapy as a python module,
when I run

jsnap.check(cfg, "PRE", "POST")

or

xx = jsnap.check(cfg, "PRE", "POST")

there are messages in standard output being printed (I am mostly interested in the error messages), for example,

*************************** Device: nl-srk03a-ri1 ***************************
Tests Included: show_bgp_summary 
************************* Command: show bgp summary *************************
PASS | All "peer-count" is greater than 1" [ 1 matched ]
PASS | All "peer-address" is same in pre and post snapshot [ 26 matched ]
PASS | All "peer-as" is same in pre and post snapshot [ 26 matched ]
PASS | All "flap-count" is same in pre and post snapshot [ 26 matched ]

ERROR: the BGP peer __.__.__.__ (ASN __) is not in Established state.
ERROR: the BGP peer __:__:__:__::__:__ (ASN __) is not in Established state.
ERROR: the BGP peer __:__:__::__:__:__ (ASN __) is not in Established state.
FAIL | All "peer-state" is not equal to "Established" [ 23 matched / 3 failed ]

PASS | All "name" is same in pre and post snapshot [ 97 matched ]
PASS | All "name" in pre snapshot is present in post snapshot [ 97 matched ]
PASS | All "name" in post snapshot is present in pre snapshot [ 97 matched ]
PASS | All "active-prefix-count" is with in delta difference of 20% [ 97 matched ]
PASS | All "received-prefix-count" is with in delta difference of 20% [ 97 matched ]
PASS | All "accepted-prefix-count" is with in delta difference of 20% [ 97 matched ]
PASS | All "suppressed-prefix-count" is with in delta difference of 20% [ 97 matched ]

If I look into xx[0].test_details['show bgp summary'] there is no such error message,
could you please advise on how-to retrieve the error messages ?

Many Thanks.
Nikos

[vnitinv]

Hi @nskalis As of today we don't provide the message as part of json data being returned. Adding messages to json was making json look very odd (even with pretty print). Hence we removed it.

[nskalis]

@vnitinv thanks for looking into it, although i did not update jsnapy today.
i would like to use jsnapy as a module, in order to write an ansible plugin.

so in that case, if a test case fails, and we do not know what is the error message or where exactly it failed, why ti would be useful ?

[eeishaan]

Hi @nskalis

If you want the formatted error string while using module version, you can use the following code.

import sys
from StringIO import StringIO
old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()
sys.stderr = mystdout
 
from jnpr.jsnapy import SnapAdmin
from pprint import pprint
from jnpr.junos import Device
 
js = SnapAdmin()
...
snapvalue = js.snapcheck(config_file, "snap")
sys.stdout = old_stdout
print(mystdout.getvalue())

Replace the snapcheck function with check function. You will have to parse the string obtained from mystdout.getvalue to extract error messages. This can be done using regex.

If you don't want to do that, you can apply the patch from this pull request . This is not merged into master because of the reasons mentioned above by @vnitinv. This also doesn't suit your needs to extract element level error messages as only top level messages are captured by the pull request

[nskalis]

@eeishaan @vnitinv hmm.. but I can follow the same approach using the original JSNAP utility, which by the way honours the err and info messages instead of mentioning a dict-key.

If you could work on it and improve JSNAPy in the user-friendliness that would be great. Also, I have opened 2-3 another issues 1 month ago that are still in Open status. I am working already with the JSNAP version of my configuration file. I would like JSNAP to be more delicate in the stdout messages and display only the error messages, but unfortunately this is not valid at the moment. On the other hand, JSNAPy is not better at that.

[eeishaan]

@nskalis Duly noted. We understand that our response dictionary is not the prettiest thing and difficult to use. We will be addressing this in upcoming releases.

[sidhujasminder]

@vnitinv #254 does it resolve this issue.

[dharpatel]

I was having the err message issue. Below is my test file and as you can see, I have mentioned err message in it. But Jsnapy is not printing that error message instead I am getting error message: Value of 'admin-status' not 'no-diff' at 'physical-interface' with {"admin-status": ["up"]}., this message has no meaning at all. I wanted Jsnapy print error message as I have described, like Interface ae110 was ['up'], now it's ['down']. If I mention Interface {{pre["name"]}} was {{pre["admin-status"]}}, now it's {{post["admin-status"]}} in info: section then its printing the correct exact same message, just that somehow err: is not working and overwriting custom error message with some other message.
has this issue already got fixed, if yes than what version of Jsnapy has fixed this issue. If not then whats the best way to work around on this.

Interface-state-Check:
- command: show interfaces terse
- iterate:
    id: name
    tests:
    - err: ' Interface {{pre["name"]}} was {{pre["admin-status"]}}, now it's {{post["admin-status"]}}'
      info: Interface State check
      no-diff: admin-status
    xpath: physical-interface