Stack Overflow
Stack Overflow

Questions tagged [confluent-platform]

a Streaming Platform providing enterprise event streaming functionalities complementing Apache Kafka.

Confluent Platform provides enterprise scaled event streaming expanding . It adds specific server infrastructure, network operability, storage solutions, administrative tools, and deployment staging, among others.

Official website: www.confluent.io

1900 questions
47
votes
1 answer

confluent platform vs apache kafka

I am new to kafka and I am curious about the Confluent platform. It seems that there are not many user stories on the Confluent platform. What are the differences between the Confluent platform and Apache Kafka?
Jihun No
  • 1,039
  • 1
  • 9
  • 23
31
votes
6 answers

Confluent Maven repository not working?

I need to use the Confluent kafka-avro-serializer Maven artifact. From the official guide I should add this repository to my Maven pom confluent http://packages.confluent.io/maven/ The problem is…
gvdm
  • 2,373
  • 4
  • 29
  • 59
25
votes
2 answers

Ideal value for Kafka Connect Distributed tasks.max configuration setting?

I am looking to productionize and deploy my Kafka Connect application. However, there are two questions I have about the tasks.max setting which is required and of high importance but details are vague for what to actually set this value to. If I…
PhillipAMann
  • 817
  • 10
  • 19
22
votes
3 answers

Connect to Kafka running in Docker

I setup a single node Kafka Docker container on my local machine like it is described in the Confluent documentation (steps 2-3). In addition, I also exposed Zookeeper's port 2181 and Kafka's port 9092 so that I'll be able to connect to them from a…
Sasha Shpota
  • 7,441
  • 6
  • 49
  • 100
20
votes
1 answer

How to pass parameters for a specific Schema registry when using Kafka Avro Console Consumer?

I am trying to use Confluent kafka-avro-console-consumer, but how to pass parameters for Schema Registry to it?
Joe
  • 8,787
  • 14
  • 63
  • 129
18
votes
3 answers

Kafka setup with docker-compose

Hi I'm currently setting up Kafka with Docker. I've managed to setup Zookeeper and Kafka with the published confluent image, see following docker-compose file: version: '2' services: zookeeper: image: confluentinc/cp-zookeeper:3.2.0 …
Omar Andres Olivares Rodriguez
  • 407
  • 2
  • 5
  • 11
17
votes
3 answers

How to have KafkaProducer to use a mock Schema Registry for testing?

I'm using KafkaProducer in my test cases and my producer uses the schemaRegistryUrl which points a my local instance of Schema Registry. Is there a way to mock how KafkaProducer connects with the Schema Registry? That is, to have…
Glide
  • 17,559
  • 21
  • 70
  • 114
13
votes
2 answers

Does Kafka python API support stream processing?

I have used Kafka Streams in Java. I could not find similar API in python. Do Apache Kafka support stream processing in python?
user3126637
  • 133
  • 1
  • 1
  • 5
13
votes
2 answers

KafkaAvroDeserializer does not return SpecificRecord but returns GenericRecord

My KafkaProducer is able to use KafkaAvroSerializer to serialize objects to my topic. However, KafkaConsumer.poll() returns deserialized GenericRecord instead of my serialized class. MyKafkaProducer KafkaProducer producer; …
Glide
  • 17,559
  • 21
  • 70
  • 114
12
votes
6 answers

Start Confluent Schema Registry in windows

I have windows environment and my own set of kafka and zookeeper running. To use custom objects, I started to use Avro. But I needed to get the registry started. Downloaded Confluent platform and ran this: $ ./bin/schema-registry-start…
user1860447
  • 1,128
  • 5
  • 18
  • 41
10
votes
1 answer

Kafka consumer gets stuck after exceeding max.poll.interval.ms

When the consumer does not receives a message for 5 mins (default value of max.poll.interval.ms 300000ms) the consumer comes to a halt without exiting the program. The consumer process hangs and does not consume any more messages. The following…
esquarer
  • 303
  • 3
  • 12
10
votes
1 answer

How to Consume from specific TopicPartitionOffset with Confluent.Kafka in .Net

I need my consumer to consume from an specific TopicPartitionOffset(here from offset 278). Suppose that Messages have been produced by some Producer in Specific topic like ="Test_1" before. Here is my Code using System; using…
M.H.Ahmadi
  • 456
  • 3
  • 12
10
votes
4 answers

AccessDeniedException when deleting a topic on Windows Kafka

I just installed Kafka (from Confluent Platform) on my Windows machine. I started up Zookeeper and Kafka and creating topics, producing to and consuming from them works. However, as soon as I delete a topic, Kafka crashes like this: PS…
rabejens
  • 6,290
  • 8
  • 35
  • 77
9
votes
1 answer

Kafka producer difference between flush and poll

We have a Kafka consumer which will read messages and do so stuff and again publish to Kafka topic using below script producer config : { "bootstrap.servers": "localhost:9092" } I haven't configured any other configuration like…
shakeel
  • 631
  • 1
  • 6
  • 20
9
votes
2 answers

Kafka connect cluster setup or launching connect workers

I am going through kafka connect, and i am trying to get the concepts. Let us say I have kafka cluster (nodes k1, k2 and k3) setup and it is running, now i want to run kafka connect workers in different nodes say c1 and c2 in distributed mode. Few…
Kalaiselvam M
  • 877
  • 1
  • 14
  • 21
1
2 3
…
99 100