Juniper admin , firewall & idps logs are not parsed correctly

[imsidr]

Note: If your issue is not a bug or a feature request, please raise a support ticket through our support portal (Splunk.com > Support > Support Portal). This will help us resolve your issue more efficiently and provide you with better assistance. For more information on how to work with the Splunk Support, please refer to this guide.

Was the issue replicated by support? no
What is the sc4s version ? 3.27.0

Which operating system (including its version) are you using for hosting SC4S? Ubuntu

Which runtime (Docker, Podman, Docker Swarm, BYOE, MicroK8s) are you using for SC4S? docker

Is there a pcap available? If so, would you prefer to attach it to this issue or send it to Splunk support? yes

Is the issue related to the environment of the customer or Software related issue? Software related issue

Is it related to Data loss, please explain ? No
Protocol? Hardware specs?

Last chance index/Fallback index? sc4s

Is the issue related to local customization? yes

Do we have all the default indexes created? yes

Describe the bug
Juniper admin , firewall & idps logs are not parsed correctly

To Reproduce
Steps to reproduce the behavior:

1. Go to '...' index=firewall or index=juniper*
2. Click on '....'
3. Scroll down to '....'
4. See error

[imsidr]

pcap attached to splunk support case # CASE [3621290]

[imsidr]

Hi @rjha-splunk @sbylica-splunk any update on this ?

[sbylica-splunk]

Hi @imsidr, looking into it, I will post an update once we have something more.

[sbylica-splunk]

@imsidr so we decided to add parsing logs with an RT_SYSTEM tag, with a sourcetype juniper:legacy. Does that seem good to you?

[imsidr]

Hi Szymon, Are you saying all the logs would have juniper:legacy sourcetype? //BR,Sid From: Szymon Bylica ***@***.***> Sent: Monday, December 2, 2024 3:04 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!tJmMmrrTUzzsub-hIqpBUxTprJmM2N1-GL6jYmgqfN3KJ0Rcd5Rr42SldvKpJu9XNrXIS9soFu-o2F-3W_uKCs7CXLOX4-wTTQ$> so we decided to add parsing logs with an RT_SYSTEM tag, with a sourcetype juniper:legacy. Does that seem good to you? — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2511014561__;Iw!!JJ-tOIoKdBzLSfV5jA!tJmMmrrTUzzsub-hIqpBUxTprJmM2N1-GL6jYmgqfN3KJ0Rcd5Rr42SldvKpJu9XNrXIS9soFu-o2F-3W_uKCs7CXLNLsUspYg$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX2CPJYJKS3KHMCYR532DQSP7AVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJRGAYTINJWGE__;!!JJ-tOIoKdBzLSfV5jA!tJmMmrrTUzzsub-hIqpBUxTprJmM2N1-GL6jYmgqfN3KJ0Rcd5Rr42SldvKpJu9XNrXIS9soFu-o2F-3W_uKCs7CXLOA54AbAw$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[sbylica-splunk]

@imsidr yeah, logs with RT_SYSTEM tag would be assigned to a juniper:legacy sourcetype.

[imsidr]

But I can see none of them are parsed.. for example I see RT_FLOW logs are not parsed. //BR,Sid From: Szymon Bylica ***@***.***> Sent: Monday, December 2, 2024 6:15 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!s7HonRGluNMysALLju91aOeZlRwD7gHzK2yaa6ZeG9ajXfwizRIsD-ag5KsVWJcktL4BNWn0r9hPtAN42VKJqijSer8On99ybg$> yeah, logs with RT_SYSTEM tag would be assigned to a juniper:legacy sourcetype. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2511444491__;Iw!!JJ-tOIoKdBzLSfV5jA!s7HonRGluNMysALLju91aOeZlRwD7gHzK2yaa6ZeG9ajXfwizRIsD-ag5KsVWJcktL4BNWn0r9hPtAN42VKJqijSer_0-7Ikhg$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX36RBDFWXJIZ6ZNRP32DRI3NAVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJRGQ2DINBZGE__;!!JJ-tOIoKdBzLSfV5jA!s7HonRGluNMysALLju91aOeZlRwD7gHzK2yaa6ZeG9ajXfwizRIsD-ag5KsVWJcktL4BNWn0r9hPtAN42VKJqijSer9yWZKlHA$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[sbylica-splunk]

@imsidr can you attach an example of RT_FLOW log that is not being parsed properly? I don't see any in the pcap file that was attached.
Logs with RT_FLOW tag should be parsed correctly since we have a rule for that.

[imsidr]

I asked Ravi from support to setup a call to show this issue, can we connect on this ? I am sure all sorts of logs were captured in both the pcaps , I am not sure how I capture it for a particular log. //BR,Sid From: Szymon Bylica ***@***.***> Sent: Wednesday, December 4, 2024 5:58 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!s8_bMQv-sPJ0aAksfaKi7k0qNUtU3RAuH_VOkbTFQtLqpWcyNrYbSsEzitM2iZtuFxuYiihQI3tqV_JJkvU70kG5ZLf_xTQQnQ$> can you attach an example of RT_FLOW log that is not being parsed properly? I don't see any in the pcap file that was attached. Logs with RT_FLOW tag should be parsed correctly since we have a rule for that. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2517211402__;Iw!!JJ-tOIoKdBzLSfV5jA!s8_bMQv-sPJ0aAksfaKi7k0qNUtU3RAuH_VOkbTFQtLqpWcyNrYbSsEzitM2iZtuFxuYiihQI3tqV_JJkvU70kG5ZLdUnM0YCw$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX4GSSR46Y2TGTPQQJT2D3YLNAVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJXGIYTCNBQGI__;!!JJ-tOIoKdBzLSfV5jA!s8_bMQv-sPJ0aAksfaKi7k0qNUtU3RAuH_VOkbTFQtLqpWcyNrYbSsEzitM2iZtuFxuYiihQI3tqV_JJkvU70kG5ZLekQaW9lQ$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[sbylica-splunk]

@imsidr Yeah, we can setup a call. What would be a possible time for it?

[imsidr]

Can we have it today ? I am available for next 4hrs. From: Szymon Bylica ***@***.***> Sent: Thursday, December 5, 2024 1:58 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!pl0otsPtRndVzUfpZtR5FZibZjn5zfoMEeleAwMViFdygujNSM0ShANRjGod0iSBbx4HhMcD7C2DC_PfPof4RMocOogiDA-_3Q$> Yeah, we can setup a call. What would be a possible time for it? — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2519604844__;Iw!!JJ-tOIoKdBzLSfV5jA!pl0otsPtRndVzUfpZtR5FZibZjn5zfoMEeleAwMViFdygujNSM0ShANRjGod0iSBbx4HhMcD7C2DC_PfPof4RMocOoi11yIWQA$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX5FBHZCKVHYNBTZ3532EAFBZAVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJZGYYDIOBUGQ__;!!JJ-tOIoKdBzLSfV5jA!pl0otsPtRndVzUfpZtR5FZibZjn5zfoMEeleAwMViFdygujNSM0ShANRjGod0iSBbx4HhMcD7C2DC_PfPof4RMocOojppIFNtg$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[sbylica-splunk]

@imsidr today probably not, could we arrange for it tomorrow or next week?

[imsidr]

Sure , we can connect tomorrow , I am available between 3-7 PM IST & 9-11PM IST //BR,Sid From: Szymon Bylica ***@***.***> Sent: Thursday, December 5, 2024 4:11 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!pMzqBx9OjLF_wiOELT3rjKkszKJBJLGFA0vN9l352KDMszTcTuaKO4uEH2Kw4AyL8NnL84NJWOWOkPJFRWgG0HkJLQAGJ9bPfg$> today probably not, could we arrange for it tomorrow or next week? — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2519936105__;Iw!!JJ-tOIoKdBzLSfV5jA!pMzqBx9OjLF_wiOELT3rjKkszKJBJLGFA0vN9l352KDMszTcTuaKO4uEH2Kw4AyL8NnL84NJWOWOkPJFRWgG0HkJLQB5eQF0xg$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX372QAHNR32FJJR6B32EAUTHAVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJZHEZTMMJQGU__;!!JJ-tOIoKdBzLSfV5jA!pMzqBx9OjLF_wiOELT3rjKkszKJBJLGFA0vN9l352KDMszTcTuaKO4uEH2Kw4AyL8NnL84NJWOWOkPJFRWgG0HkJLQAg7ZORRw$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[sbylica-splunk]

Hey @imsidr, so we have two follow-up questions for now:

• Can we get a .pcap file with an example of logs with RT_FLOW tag, like the one we saw on a meeting with the customer?
• Were the logs getting parsed earlier? Was there any noticeable trigger point that led to this issue?

[imsidr]

Hi Szymon , I have attached the pcap in spluk support case. //BR,Sid From: Szymon Bylica ***@***.***> Sent: Wednesday, December 11, 2024 3:01 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) Hey @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!oMKnsmnGeILH8pITmDFWMv1Iiarkq2cIsDJ8rH9GWK8gFN-cpy-gKYgBhMuFzm_bDNZTRyeJ-wqsVU8oSkxeMlQ8H8gr6-1zUQ$>, so we have two follow-up questions for now: * Can we get a .pcap file with an example of logs with RT_FLOW tag, like the one we saw on a meeting with the customer? * Were the logs getting parsed earlier? Was there any noticeable trigger point that led to this issue? — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2535302054__;Iw!!JJ-tOIoKdBzLSfV5jA!oMKnsmnGeILH8pITmDFWMv1Iiarkq2cIsDJ8rH9GWK8gFN-cpy-gKYgBhMuFzm_bDNZTRyeJ-wqsVU8oSkxeMlQ8H8ikEbPAaA$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX5G5KDBQ7JEEYNGSVT2FAA6HAVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZVGMYDEMBVGQ__;!!JJ-tOIoKdBzLSfV5jA!oMKnsmnGeILH8pITmDFWMv1Iiarkq2cIsDJ8rH9GWK8gFN-cpy-gKYgBhMuFzm_bDNZTRyeJ-wqsVU8oSkxeMlQ8H8gtt6911w$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[sbylica-splunk]

@imsidr thanks, could you also ask for a screenshot from the Splunk side? We would like to see the message and the sc4s_tags field.

[imsidr]

I have attached screenshots in the case. From: Szymon Bylica ***@***.***> Sent: Wednesday, December 11, 2024 5:03 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!umd3ZisvGGrnsxFoZk-holQ3kIFKUZT5jjHEqA89GD8_RdvDSf9AoWoLa64bs0xJaf87_uJS_Fo6GmpVXDyDCDznf7NCSfNbGA$> thanks, could you also ask for a screenshot from the Splunk side? We would like to see the message and the sc4s_tags field. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2535640024__;Iw!!JJ-tOIoKdBzLSfV5jA!umd3ZisvGGrnsxFoZk-holQ3kIFKUZT5jjHEqA89GD8_RdvDSf9AoWoLa64bs0xJaf87_uJS_Fo6GmpVXDyDCDznf7NMToaa2Q$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX2TPKK2W4FRZ3YW5VT2FAPIJAVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZVGY2DAMBSGQ__;!!JJ-tOIoKdBzLSfV5jA!umd3ZisvGGrnsxFoZk-holQ3kIFKUZT5jjHEqA89GD8_RdvDSf9AoWoLa64bs0xJaf87_uJS_Fo6GmpVXDyDCDznf7PAfLkxUA$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[sbylica-splunk]

Hi @imsidr, maybe I missed it but was there any response to this question?
Were the logs getting parsed earlier? Was there any noticeable trigger point that led to this issue?

Also, a new version of SC4S with the fix for RT_SYSTEM logs was released, can the customer upgrade to this version?
https://github.com/splunk/splunk-connect-for-syslog/releases/tag/v3.33.0

[imsidr]

Hi Szymon, It was being parsed when we were onboarding it using legacy method. Also we have Juniper Add-on installed with some additional configs from out side but it is not related to parsing. Also the issue is not related to RT_SYSTEM only , none of the events are parsed. But I will still go ahead and upgrade it. //BR,Sid From: Szymon Bylica ***@***.***> Sent: Thursday, December 12, 2024 6:58 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) Hi @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!otXOrer8wBw-0jD4pXW9EIZB3dEkZw_bxDnV4uyuQmrn1RUQaYuezx60RkAuQG6iJhN119ON9vBNnZ2YDOvXHBCs3WE_pvSdNg$>, maybe I missed it but was there any response to this question? Were the logs getting parsed earlier? Was there any noticeable trigger point that led to this issue? Also, a new version of SC4S with the fix for RT_SYSTEM logs was released, can the customer upgrade to this version? https://github.com/splunk/splunk-connect-for-syslog/releases/tag/v3.33.0<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/releases/tag/v3.33.0__;!!JJ-tOIoKdBzLSfV5jA!otXOrer8wBw-0jD4pXW9EIZB3dEkZw_bxDnV4uyuQmrn1RUQaYuezx60RkAuQG6iJhN119ON9vBNnZ2YDOvXHBCs3WEzXBqVFw$> — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2538920279__;Iw!!JJ-tOIoKdBzLSfV5jA!otXOrer8wBw-0jD4pXW9EIZB3dEkZw_bxDnV4uyuQmrn1RUQaYuezx60RkAuQG6iJhN119ON9vBNnZ2YDOvXHBCs3WFHbV7MKw$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX7SC2KSU5DSGDR7RUD2FGFNHAVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZYHEZDAMRXHE__;!!JJ-tOIoKdBzLSfV5jA!otXOrer8wBw-0jD4pXW9EIZB3dEkZw_bxDnV4uyuQmrn1RUQaYuezx60RkAuQG6iJhN119ON9vBNnZ2YDOvXHBCs3WECMuxbrQ$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[sbylica-splunk]

Hi @imsidr, So, it was being parsed correctly using the legacy method? Do we have a configuration/description of this method? Ok, let's see if updating fixes this issue, keep me updated.

[imsidr]

We are using syslog-ng relay to get the data and forward it to the splunk cloud, we have juniper TA(customized) which is doing all the parsing & extraction stuff , I will attach the copy on support case. //BR,Sid From: Szymon Bylica ***@***.***> Sent: Friday, December 13, 2024 3:23 PM To: splunk/splunk-connect-for-syslog ***@***.***> Cc: Rai, Siddhartha ***@***.***>; Mention ***@***.***> Subject: Re: [splunk/splunk-connect-for-syslog] Juniper admin , firewall & idps logs are not parsed correctly (Issue #2643) Hi @imsidr<https://urldefense.com/v3/__https:/github.com/imsidr__;!!JJ-tOIoKdBzLSfV5jA!tOLxBRTxVvuNhPaStQCEy14UtlNMzG4-bhVIDjbjSOEF_QzuUXgTjdLZCmc0075oZNEaqVxWF36U00nrMyBCNjYa3stqY5h5FQ$>, So, it was being parsed correctly using the legacy method? Do we have a configuration/description of this method? Ok, let's see if updating fixes this issue, keep me updated. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/splunk/splunk-connect-for-syslog/issues/2643*issuecomment-2541025418__;Iw!!JJ-tOIoKdBzLSfV5jA!tOLxBRTxVvuNhPaStQCEy14UtlNMzG4-bhVIDjbjSOEF_QzuUXgTjdLZCmc0075oZNEaqVxWF36U00nrMyBCNjYa3ssadf2LyA$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BETPJX4YYJQTISHQA5TEUVD2FKVBRAVCNFSM6AAAAABSCRG5TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBRGAZDKNBRHA__;!!JJ-tOIoKdBzLSfV5jA!tOLxBRTxVvuNhPaStQCEy14UtlNMzG4-bhVIDjbjSOEF_QzuUXgTjdLZCmc0075oZNEaqVxWF36U00nrMyBCNjYa3svtIFI92A$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[imsidr]

@sbylica-splunk @sbylica-splunk we are seeing jinja2 module errors after upgrading to v 3.33 , the container is in restarting status , we already have jinja2 module installed , not sure why are we getting this error. Attaching sc4s file in the case.