1

I'm trying to build a system for a transportation company. The company has many trucks and deliver items to different countries. One of the use cases of the system is to track each truck and view it's position in real time. So, each truck send GPS data to Kafka topic using "truck_id" as key. Data is send every 1 second.

Let say that data is JSON like this:

{
    "truck_id": "STU_28",
    "plate": "ZH1234Z"
    "lat": 47.3688196,
    "lng": 8.5204119
}

How can I send these data to client app? This is the part that I get stuck. There are 3 clients, web client, android client and ios client.

This is the use case: one user opens the web client and selects 3 trucks from trucks list. Then clicks a button to display each truck real time position on map(I'm using OpenStreetMap and Leaflet to show the map).

I have search about this and found that I could use websockets for sending data from server to client. I build a NodeJS server using websocket module but those connections are stored in MemoryStore. What I want is to store those connections somewhere else that I can get later. I found some solutions using Redis as PUB/SUB but those solutions where using Socket.IO, I'm using websocket module. I know that this use case is kind a publish/subscribe model.

So trucks publish data(their positions) and users subscribe to those data(to display on map). The problem is that there are different "groups" of subscribers, for example:

"user1" wants to show location of trucks ["STU_28", "STU_818", "ZFL_00_1"],

"user2" wants to show location of just one truck,

"user3" wants to show location of 5 other trucks etc.

When user navigates to another page or logs out from web app the websocket connection needs to terminate. So I have to remove the connection from store. Also this service(nodejs or another) acts as a Kafka consumer, gets data from Kafka topic and send it to users app.

A pseudo code could be like this:

KafkaConsumer consumer = new KafkaConsumer();
        
consumer.subscribe(topicName);
        
while(true) {
    // position is json data like above which contains "truck_id"
    position = consumer.poll();

    // go to websocket connection store and get all users(connections) that had searched(subscribed) to that "truck_id"
    connections = wsConnectionStore.getConnections(position.truck_id);
    
    for(WSConnection connection : connections) {
        connection.sendUTF(position);
    }
}

How can I achieve this? No necessary to be in nodejs, it can be in java, python.

Note: the system needs to be scalable and highly available.

Community
  • 1
  • 1
bourne
  • 11
  • 2
  • Hello and welcome to Stackoverflow. To me this question is far too broad to be answered here on SO. Please have a look at [How to ask a good question](https://stackoverflow.com/help/how-to-ask) and ideally narrow your question to a specific programming question, ideally with a [minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example) – mike May 06 '20 at 06:27
  • @bourne Did you find any approach for this? – Hmmm May 22 '20 at 07:35
  • @Hmmm Unfortunately not yet. – bourne Nov 03 '20 at 21:45
  • @bourne ... so I have manged to do it by adding some of the user info which is required to filter out the data we need to send him, and for each of the notification I need to send I loop through connection just like you are doing above and send the filtered data on that particular channel – Hmmm Nov 04 '20 at 06:25

0 Answers0