Stack Overflow
Stack Overflow

Questions tagged [nsq]

NSQ is a realtime message processing system designed to operate at bitly’s scale, handling billions of messages per day.

NSQ is a realtime message processing system designed to operate at bitly’s scale, handling billions of messages per day.

It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee.

Operationally, NSQ is easy to configure and deploy (all parameters are specified on the command line and compiled binaries have no runtime dependencies). For maximum flexibility, it is agnostic to data format (messages can be JSON, MsgPack, Protocol Buffers, or anything else). Go and Python libraries are available out of the box.

Questions can also be posted to the nsq-users Google group.

Bugs can be filed on GitHub.

55 questions
1
vote
1 answer

Which --broadcast-address should i set in nsqd param?

Now i am starting nsqd with nsqd --max-msg-timeout=1800000 --lookupd-tcp-address=127.0.0.1:4160 I am using nsqjd for nodejs with nsq. There was everyhing ok until some days. Now, when starting nodejs with nsqjs in it i receive error: Error:…
Tyler Brin
  • 49
  • 4
1
vote
1 answer

Is the TimerThread cause DeadLock happend?

Timer-2 thread is waiting on the 0x00000000e1a23398, and it's also locking the 0x00000000e1a23398. Will this situation cause Timer-2 thread deadlock? Can anybody explain it for me?
Kay
  • 59
  • 6
1
vote
2 answers

NSQ Re-Queue half Message

I have an array of Ids of type int64 And this is my Nsq Message that I am trying to publish. nsqMsg := st{ Action : "insert", Ids : Ids GID : Gids } msg, err := json.Marshal(nsqMsg) if err != nil { …
Kartik Khandelwal
  • 11
  • 1
1
vote
0 answers

Why were the messages delayed in NSQ? It's unexpected……

I used the NSQ as the mq in my project，java producer produce the message to NSQ and go consumer consume it.But the strange things is that the consumer always get the message after few seconds.There is just a few messages,I really don't know how to…
Siyong Xiao
  • 21
  • 3
1
vote
0 answers

Move Data from Elastic Search to NSQ and then to a Mysql database

I have a use case where we are using snowplow analytics to get data from various api's and webpages , due to some restrictions(snowplow) we are only able to get the data to ES through a NSQ channel. Is there any way to move data from Elastic Search…
INFOSYS
  • 1,137
  • 6
  • 17
  • 36
1
vote
1 answer

nsqjs clients, not receiving messages immediately from go-nsq server side

trying to learn nsq, and following the examples from here golang example and here nsqjs. I am sending messages in server side doing w/ a for loop and go routines var wg sync.WaitGroup for i := 0; i < 100; i++ { wg.Add(1) go func(x int) { …
Hokutosei
  • 1,773
  • 4
  • 19
  • 40
1
vote
1 answer

How to send the message back to the sender with messaging architecture?

I'm solving the communications between the microservices with messaging-architecture. Let's say I have a tradition application, and there're User, Post Video modules. You can create the posts, videos with it, but before that, I need to convert the…
Yami Odymel
  • 1,495
  • 1
  • 18
  • 35
1
vote
2 answers

Kubernetes get endpoints

I have a set of pods providing nsqlookupd service. Now I need each nsqd container to have a list of nsqlookupd servers to connect to (while service will point to different every time) simultaneously. Something similar I get with kubectl describe…
Dmytro Leonenko
  • 1,333
  • 4
  • 16
  • 30
1
vote
0 answers

use environment variable to look for HTTPAddresses

I am trying to use an environment variable to not set the "http address", because the IP address may change while use to AWS machine. How can I use an environment variable where it find the HTTPAddresses? Here I use the localhost address but at AWS…
Istiak Mahmood
  • 2,401
  • 7
  • 17
  • 46
1
vote
2 answers

nsq go client can't keep up

I'm a Go novice coming from the Node world and I'm building a consumer using the official Bitly Go client. I am using AddConcurrentHandlers to spawn 50 goroutines to handle the fire hose of messages. The issue is that my consumer falls behind…
Walter Cecil Worsley IV
  • 334
  • 1
  • 8
  • 19
1
vote
2 answers

How to connect nsqd and nsqlookupd containers?

I am trying to connect these 2 docker containers: nsqd: https://registry.hub.docker.com/u/mreiferson/nsqlookupd/ nsqlookupd: https://registry.hub.docker.com/u/mreiferson/nsqlookupd/ These are the official docker containers for nsqd and…
rexposadas
  • 2,689
  • 3
  • 24
  • 46
1
vote
0 answers

pynsq: Giving up RDY count

WHAT: Could someone help me understand what this error message means. nsqadmin shows a lot of the messages being requeued / timed out on . Could these logs be the reason? I am using pynsq 0.4.2 STACKTRACE 2013-07-30 13:34:18,827 DEBUG…
Sagar Hatekar
  • 8,348
  • 14
  • 52
  • 72
0
votes
0 answers

nsqjs dependency error: bignumber is not a constructor

I have added nsqjs as a dependency but my build fails because of following error TypeError: BigNumber is not a constructor BigNumber.js official documentation says it should be imported as const BigNumber = require('bignumber.js'); but in…
Ishan Ghorela
  • 21
  • 2
0
votes
0 answers

Flask + nsq.run() via threding -> ValueError: signal only works in main thread

Try to start Flask and nsq reader in one process. I get an error ValueError: signal only works in main thread How to fix it? List 1 -- run command and error List 2 -- my python file List 1. execution (env) (base) user@vaio:~/Documents/app$…
LovingFox
  • 23
  • 4
0
votes
0 answers

nsq max message size and max body size difference

I saw these on nsq setting configuration -max-body-size int maximum size of a single command body (default 5242880) -max-msg-size int maximum size of a single message in bytes (default 1048576) what's actually the difference between…
DemiDust
  • 221
  • 2
  • 14
1
2
3 4