SQS

Spout for SQS

Bases: Spout

__init__(output, state, **kwargs)

Initialize the SQS class.

Parameters:

Name	Type	Description	Default
output	StreamingOutput	An instance of the StreamingOutput class for saving the data.	required
state	State	An instance of the State class for maintaining the state.	required
**kwargs		Additional keyword arguments.	{}

Using geniusrise to invoke via command line

genius SQS rise \
    streaming \
        --output_kafka_topic sqs_test \
        --output_kafka_cluster_connection_string localhost:9094 \
    none \
    listen \
        --args queue_url=https://sqs.us-east-1.amazonaws.com/123456789012/my-queue batch_size=10 batch_interval=10

Using geniusrise to invoke via YAML file

version: "1"
spouts:
    my_sqs_spout:
        name: "SQS"
        method: "listen"
        args:
            queue_url: "https://sqs.us-east-1.amazonaws.com/123456789012/my-queue"
            batch_size: 10
            batch_interval: 10
        output:
            type: "streaming"
            args:
                output_topic: "sqs_test"
                kafka_servers: "localhost:9094"

listen(queue_url, batch_size=10, batch_interval=10)

📖 Start listening for new messages in the SQS queue.

Parameters:

Name	Type	Description	Default
queue_url	str	The URL of the SQS queue to listen to.	required
batch_size	int	The maximum number of messages to receive in each batch. Defaults to 10.	10
batch_interval	int	The time in seconds to wait for a new message if the queue is empty. Defaults to 10.	10

Raises:

Type	Description
Exception	If unable to connect to the SQS service.