Stack Overflow
Stack Overflow

Questions tagged [nsq]

NSQ is a realtime message processing system designed to operate at bitly’s scale, handling billions of messages per day.

NSQ is a realtime message processing system designed to operate at bitly’s scale, handling billions of messages per day.

It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee.

Operationally, NSQ is easy to configure and deploy (all parameters are specified on the command line and compiled binaries have no runtime dependencies). For maximum flexibility, it is agnostic to data format (messages can be JSON, MsgPack, Protocol Buffers, or anything else). Go and Python libraries are available out of the box.

Questions can also be posted to the nsq-users Google group.

Bugs can be filed on GitHub.

55 questions
0
votes
0 answers

Can nsq topics be split across nsqd instances?

I am considering using nsq in a docker swarm setup and I'm trying to get a better understanding of whether or not I need to constrain certain services that write to topics to using a specific nsqd or if I can let them write to any available nsqd. My…
Joe Phillips
  • 44,686
  • 25
  • 93
  • 148
0
votes
1 answer

Attempt to reconnect to NSQ after clean close

I've created an NSQ producer in my Go microservice and I'm trying to create a method for it to attempt to reconnect after an NSQ restart. I've looked into the nsq config documentation hoping to see if I could provide a failover method on…
Jamie Belcher
  • 113
  • 2
  • 13
0
votes
1 answer

How to get a return value of a function in python

import gnsq class something(): def __init__(self, pb=None, pk=None, address=None): self.pb = pb self.pk = pk self.address = address def connect(self): consumer = gnsq.Consumer(self.pb, 'ch', self.address) …
swoopy
  • 99
  • 12
0
votes
1 answer

depth of channel showing in nsqadmin increasing fast when producer send message to same topic via different nsqd

I play with nsqd a little bit and met the prob mentioned in the title. But when I send message via single nsqd, there is no such prob. Does this mean message should always be sent via same nsqd? Here is my project.…
hoozecn
  • 437
  • 3
  • 14
0
votes
1 answer

cannot connect to nsqd from another service defined in docker-compose file

I have a service connecting to nsqd to produce and consume messages. I have integration tests connecting to it (broadcast address 127.0.0.1) and this works fine executing it locally in the cli or the ide. Then I have created this service to up with…
hosseio
  • 1,000
  • 1
  • 10
  • 22
0
votes
1 answer

NSQD in docker compose

How should nsqd be used in a live microservices docker environment? Should we run nsqd in its own docker container or should each container that runs a producing service also contain an instance of nsqd? I've not managed to find any examples of how…
Paul
  • 144
  • 1
  • 6
0
votes
0 answers

Set IP address and port of Nsqlookupd in docker compose file

Is there a way to manually set the lookupd connection of the NSQd Node in the docker compose file through the command: \nsq? Below is the docker-compose file provided by nsq version: '3' services: nsqlookupd: image: nsqio/nsq command:…
vaughn ellis ramos
  • 29
  • 7
0
votes
1 answer

Ruby nsq how to listen for new messages

My setup is as follows: 1 Microservice that receives a request and writes a message in the queue(nsq) and a second microservice that must read the messages in the queue and do something based on them. I am new to the nsq concept in ruby on rails. I…
Lucian Tarna
  • 1,453
  • 2
  • 18
  • 40
0
votes
1 answer

Avoiding message timeout in pynsq

For example, I send a message to NSQ by issuing: curl -d "test2" http://127.0.0.1:4151/pub?topic=hello I've found that if message handler execution takes longer than 100 second, it will throw and this message will be timed…
jinze huang
  • 31
  • 3
0
votes
0 answers

Config nsq.io with public IP

I am using http://nsq.io/ for pub/sub in our system. Currently, we just use Nsq.io for local server only, I public a message use localhost as: $ curl -d "" http://127.0.0.1:4151/pub?topic=name I plan to use Nsq.io in public server…
Ha Doan
  • 583
  • 5
  • 17
0
votes
1 answer

Temporary NSQ Channels

I have nodes that need to connect to NSQ, they need to get all the messages for that topic. To do this, they each create a channel. I am wondering if there is a way for nsqd to clean them up after the consumers have been disconnected?
Vitaly Babiy
  • 4,896
  • 4
  • 23
  • 22
0
votes
1 answer

why nsqlookupd package use Context?

When I read the nsqlookupd part of nsq source code, I found that the author used a Context truct to wrap a NSQLookupd struct and no other funcs for Context.I don't know why we use it this way, what benefits we can get by this way? The source code…
关力宁
  • 1
0
votes
1 answer

Discovering nsqd server address from nslookupd

I'm running an nsq cluster in Docker containers using the following docker-compose.yaml file: version: '2' services: nsqlookupd: image: nsqio/nsq command: /nsqlookupd ports: - "4160" - "4161:4161" nsqd: image:…
Pyramid Newbie
  • 4,987
  • 3
  • 20
  • 26
0
votes
1 answer

Trying to create NSQ PetSet, pods keep terminating shortly after container launches

Full yaml file here (not embedded in question because it's rather long and because much of the important bits are covered by the describe below): https://gist.github.com/sporkmonger/46a820f9a1ed8a73d89a319dffb24608 Using a public container image I…
Bob Aman
  • 31,735
  • 9
  • 67
  • 95
0
votes
1 answer

getaddrinfow fails if running process from Python

I try to run 3rd party process (nsqd.exe) from my python script but when I do nsqd fails to bind socket. I've no idea why. The script I'm using: import subprocess import sys proc = subprocess.Popen(['nsqd.exe',…
David Skuza
  • 120
  • 1
  • 7
1 2
3
4