HTTP polling

Spout for HTTP polling

Bases: Spout

__init__(output, state, **kwargs)

Initialize the RESTAPIPoll class.

Parameters:

Name	Type	Description	Default
output	StreamingOutput	An instance of the StreamingOutput class for saving the data.	required
state	State	An instance of the State class for maintaining the state.	required
**kwargs		Additional keyword arguments.	{}

Using geniusrise to invoke via command line

genius RESTAPIPoll rise \
    streaming \
        --output_kafka_topic restapi_test \
        --output_kafka_cluster_connection_string localhost:9094 \
    none \
    listen \
        --args url=https://api.example.com method=GET interval=60

Using geniusrise to invoke via YAML file

version: "1"
spouts:
    my_restapi_poll:
        name: "RESTAPIPoll"
        method: "listen"
        args:
            url: "https://api.example.com"
            method: "GET"
            interval: 60
        output:
            type: "streaming"
            args:
                output_topic: "restapi_test"
                kafka_servers: "localhost:9094"

listen(url, method, interval=60, body=None, headers=None, params=None)

Start polling the REST API for data.

Parameters:

Name	Type	Description	Default
url	str	The API endpoint.	required
method	str	The HTTP method (GET, POST, etc.).	required
interval	int	The polling interval in seconds. Defaults to 60.	60
body	Optional[Dict]	The request body. Defaults to None.	None
headers	Optional[Dict[str, str]]	The request headers. Defaults to None.	None
params	Optional[Dict[str, str]]	The request query parameters. Defaults to None.	None

poll_api(url, method, body=None, headers=None, params=None)

📖 Start polling the REST API for data.

Parameters:

Name	Type	Description	Default
url	str	The API endpoint.	required
method	str	The HTTP method (GET, POST, etc.).	required
interval	int	The polling interval in seconds.	required
body	Optional[Dict]	The request body. Defaults to None.	None
headers	Optional[Dict[str, str]]	The request headers. Defaults to None.	None
params	Optional[Dict[str, str]]	The request query parameters. Defaults to None.	None

Raises:

Type	Description
Exception	If unable to connect to the REST API server.