dlt.destinations.impl.databricks.factory

databricks Objects

class databricks(Destination[DatabricksClientConfiguration,
                             "DatabricksClient"])

View source on GitHub

__init__

def __init__(credentials: Union[DatabricksCredentials, Dict[str, Any],
                                str] = None,
             is_staging_external_location: bool = False,
             staging_credentials_name: str = None,
             destination_name: str = None,
             environment: str = None,
             staging_volume_name: str = None,
             create_indexes: bool = False,
             **kwargs: Any) -> None

View source on GitHub

Configure the Databricks destination to use in a pipeline.

All arguments provided here supersede other configuration sources such as environment variables and dlt config files.

Arguments:

• credentials Union[DatabricksCredentials, Dict[str, Any], str], optional - Credentials to connect to the databricks database. Can be an instance of DatabricksCredentials or a connection string in the format databricks://user:password@host:port/database
• is_staging_external_location bool, optional - If true, the temporary credentials are not propagated to the COPY command
• staging_credentials_name str, optional - If set, credentials with given name will be used in copy command
• destination_name str, optional - Name of the destination, can be used in config section to differentiate between multiple of the same type
• environment str, optional - Environment of the destination
• staging_volume_name str, optional - Name of the staging volume to use
• create_indexes bool, optional - Whether PRIMARY KEY or FOREIGN KEY constrains should be created
• **kwargs Any - Additional arguments passed to the destination config