-1
[2020-05-05T04:27:54.668Z] "GET /click?ab_tst_bckt=-1&dvc_id=IddFVZ8W--0nuPg9P31T&featured=99&hasOfferId=1120&impressionId=NgnMfW7bYb&page_type=Search&paymentType=cpc&placement=ratetable&position=1&rf=https%3A%2F%2Fwww.ratecity.com.au%2Fcar-loans%2Flow-interest&uuid=1abb33fc-e6cd-45c6-94de-44291dfb2871&vertical=car-loans HTTP/1.1" 200 - "-" "-" 0 11971 17 17 "2001:8003:d576:b100:984a:21b6:d74e:67c7, 2001:8003:d576:b100:984a:21b6:d74e:67c7,54.206.38.58,192.168.73.159" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36" "d3632811-e50e-4686-9985-9d10d78e75ba" "swift.ratecity.com.au" "127.0.0.1:4444" inbound|4444|http|swift-client.ratecity.svc.cluster.local - 192.168.13.129:4444 192.168.73.159:0 -

Is there a grok pattern already available for this envoyproxy log?

I am using debugger on kibana and so far come this far, need help in finishing this up

\[%{GREEDYDATA:timestamp}\] \"%{WORD:method} %{URIPATH:request} ?%{URIPARAM:url_params} %{URIPROTO:protocol}/%{NUMBER:httpversion}" %{NUMBER:response} - "-" "-" %{GREEDYDATA:flags} %{IP:client_ip}(,\\s%{IP})*
Mah3ndra
  • 53
  • 11
  • You don't ask people to do your work "need help in finishing this up". if you have a specific question on an issue you should ask. otherwise delete this question please. – eladyanai May 27 '20 at 09:59
  • You should read the question properly, I asked if there is any existing patterns available. Because I see the patterns available for Nginx logs. As this is a standard thing and if one has solved it, it can help others save all the effort. I thought SO exists for the same reason to help each other. Anyway I solved this, see the answer it can help you @eladyanai – Mah3ndra May 27 '20 at 12:06

1 Answers1

0

Here is the Grok pattern you can use to pre-process envoyproxy logs before saving them to elasticsearch indices

"""\[%{GREEDYDATA:istio.timestamp}\] \"%{WORD:istio.method} %{URIPATH:istio.request}(?:%{URIPARAM:istio.url_params}|-)? %{URIPROTO:istio.protocol}/%{NUMBER:istio.httpversion}\" %{NUMBER:istio.response} (?<istio.response_flag>%{DATA}|-)? \"-\" \"-\" %{NUMBER:istio.bytes_received} %{NUMBER:istio.bytes_sent} %{NUMBER:istio.duration} (?<istio.service_time>%{NUMBER}|-)? \"(?<istio.x_forwarded_for>%{IP:istio.client_ip},%{SPACE}.+?(?=")|-)\" \"(?<istio.user_agent>%{DATA}.+?(?=")|-)\"?"""
Mah3ndra
  • 53
  • 11