AggregationExample

class kafkaaggregator.example.AggregationExample

Bases: object

Initialize topics and produce messages for the aggregation example.

The aggregation example can be used to evaluate the performance of the kafka-aggregator application.

Attributes Summary

MAX_NFIELDS
MAX_NTOPICS

Methods Summary

create_record(name)	Create a Faust-avro Record class for the source topic.
initialize(app)	Initialize source topics for the aggregation example.
make_fields()	Make fields for the example topics.
produce(app, frequency, max_messages)	Produce messages for the source topics in the aggregation example.

Attributes Documentation

MAX_NFIELDS = 999

MAX_NTOPICS = 999

Methods Documentation

create_record(name: str) → Record

Create a Faust-avro Record class for the source topic.

With a Faust-avro Record for the source topic it is possible to produce messages in Avro format for the aggregation example, instead of using value_type=bytes.

Returns:

record – Faust-avro Record class for the source topic.

Return type:

Record

async initialize(app: App) → None

Initialize source topics for the aggregation example.

To initialize the topic, its schema needs to be registered in the Schema Registry and the topic itself needs to be created in Kafka.

Parameters:

app (faust_avro.App) – Faust application

make_fields() → List[Field]

Make fields for the example topics.

Returns:

fields – A list of fields mapping field name and type.

Return type:

list

async produce(app: App, frequency: float, max_messages: int) → None

Produce messages for the source topics in the aggregation example.

In the aggregation example we can specify the frequency in which the messages are produced, the maximum number of messages for each source topic and the number of fields in every message.

Parameters:

• app (faust_avro.App) – Faust application

• frequency (float) – The frequency in Hz in wich messages are produced.

• max_messages (int) – The maximum number of messages to produce.