SQLite

Bases: Spout

__init__(output, state, **kwargs)

Initialize the SQLite class.

Parameters:

Name	Type	Description	Default
output	BatchOutput	An instance of the BatchOutput class for saving the data.	required
state	State	An instance of the State class for maintaining the state.	required
**kwargs	Any	Additional keyword arguments.	{}

Using geniusrise to invoke via command line

genius SQLite rise \
    batch \
        --output_s3_bucket my_bucket \
        --output_s3_folder s3/folder \
    none \
    fetch \
        --args s3_bucket=my_s3_bucket s3_key=mydb.sqlite query="SELECT * FROM table" page_size=100

Using geniusrise to invoke via YAML file

version: "1"
spouts:
    my_sqlite_spout:
        name: "SQLite"
        method: "fetch"
        args:
            s3_bucket: "my_s3_bucket"
            s3_key: "mydb.sqlite"
            query: "SELECT * FROM table"
            page_size: 100
        output:
            type: "batch"
            args:
                bucket: "my_bucket"
                s3_folder: "s3/folder"

fetch(s3_bucket, s3_key, query, page_size=100)

📖 Fetch data from an SQLite database and save it in batch.

Parameters:

Name	Type	Description	Default
s3_bucket	str	The S3 bucket containing the SQLite database.	required
s3_key	str	The S3 key for the SQLite database.	required
query	str	The SQL query to execute.	required
page_size	int	The number of rows to fetch per page. Defaults to 100.	100

Raises:

Type	Description
Exception	If unable to connect to the SQLite database or execute the query.