All Questions
12 questions
0
votes
2
answers
4k
views
Disable Certificate validation in SchemaRegistryClient Confluent Kafka
So, I want to read a topic from kafka (Confluent) where data lies in Avro format.
For certain unavoidable reasons , I would like to disable certificate validation.
I am using security.protocol= ...
- 990
0
votes
0
answers
3k
views
Unknown KafkaError 87 from Kafka
I am writing an Avro message to Kafka using Confluent:
import avro_validator
import avro.schema
from confluent_kafka import avro as confluent_avro
from confluent_kafka.cimpl import KafkaError, Message
...
- 755
1
vote
1
answer
892
views
Unable to write avro data to kafka using python
I'm using kafka kafka_2.11-0.11.0.2 and confluent version 3.3.0 for schema registry.
I have defined an avro schema as follows:
{
"namespace": "com.myntra.search",
"type": ...
- 7,260
0
votes
1
answer
2k
views
How to get schema from confluent schema registry with schema id and version using python
Can we pass both schema id and version to get the schema from schema registry? I know about these functions,
Getting schema by ID
sr = SchemaRegistryClient('localhost:8081')
my_schema = sr.get_by_id(...
- 127
0
votes
0
answers
487
views
Unable to produce any Kafka messages
I am new to Kafka. I was trying to run the example code stated in https://github.com/confluentinc/confluent-kafka-python/blob/master/examples/avro-cli.py.
As per the code, this is how producer and ...
- 51
0
votes
2
answers
2k
views
How can I produce without schema as avro type with python?
I use below code and send message to kafka. It works.
But I want to send message without schema because I have schema on kafka topic. I register it first. I do not want to send schema everytime.
...
- 7,366
2
votes
3
answers
8k
views
TypeError: Object of type 'mappingproxy' is not JSON serializable
I am try to publish an avro message using AvroProducer of confluent-kafka-python using schema-registry. But the code fails to serialize enum types. Below is the code and error trace. Any help is much ...
- 2,038
2
votes
2
answers
2k
views
Change schema name strategy in confluent Kafka
I am using confluent Kafka and schema registry and I cannot figure out a way to have custom schema name in python. From this blog I got how to do it in Java, but do not have any idea what the ...
- 622
1
vote
1
answer
2k
views
Confluent-Kafka: Avro Serialization Confusions with Schema Handling in Python Consumers
I am try to understand Avro Serialization on Confluent Kafka along with Schema Registry usage. It was all going well till the end but the final expectations from AVRO made lots of Confusions to me. As ...
0
votes
1
answer
502
views
How to create Transform in Memsql when source is Kafka Avro Format
I am able to push data from Kafka to Memsql.
I am trying to push using Transform. I have created Kafka Consumer in Python which is consuming data from Kafka Topic and converting to Json Format.
I ...
- 2,070
0
votes
2
answers
1k
views
Do we need to manually cache schema registry?
We are currently using Protocol Buffers as serialization mechanism for kafak message. We are going to move to Avro. We tested Avro Confluent consumer with Schema Registry and according to those tests, ...
- 611
1
vote
0
answers
789
views
Read avro messages from kafka to PySpark 2.2
I am using kafka Confluent 4.0.0 to fetch data from SQL Server into kafka topics.
I want to read topics data stored in kafka from spark streaming program using below program:
from pyspark import ...
- 21