Glue

25 resource kinds. Generated from the operator's CRD schemas.

API Group: aws.konfig.io/v1alpha1

GlueBlueprint # CLOUD CONTROL · AWS::Glue::Blueprint

GlueBlueprint manages AWS::Glue::Blueprint through the AWS Cloud Control API.

kubectl get glueblueprints · example on GitHub

Example

# GlueBlueprint — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueblueprint
apiVersion: aws.konfig.io/v1alpha1
kind: GlueBlueprint
metadata:
  name: example-glueblueprint
  namespace: default
spec:
  blueprintLocation: example
  description: Managed by konfig-konector
  name: example-glueblueprint
  providerRef:
    name: example-glueblueprint
    region: us-east-1
  tags:
  - key: example
    value: example

Spec

FieldTypeRequiredDescription
blueprintLocationstringyesSpecifies a path in Amazon S3 where the blueprint is published.
descriptionstring—A description of the blueprint.
namestringyesThe name of the blueprint.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
tagsarray<object>—The tags to be applied to this blueprint.
tags[].keystringyes
tags[].valuestringyes

Status

FieldTypeRequiredDescription
arnstring—The Amazon Resource Name (ARN) of the blueprint.
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
createdOnstring—The date and time the blueprint was registered.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastModifiedOnstring—The date and time the blueprint was last modified.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
parameterSpecstring—A JSON string that indicates the list of parameter specifications for the blueprint.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.
statusenum—The status of the blueprint registration. Allowed: CREATING, ACTIVE, UPDATING, FAILED

GlueCatalog # CLOUD CONTROL · AWS::Glue::Catalog

GlueCatalog manages AWS::Glue::Catalog through the AWS Cloud Control API.

kubectl get gluecatalogs · example on GitHub

Example

# GlueCatalog — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluecatalog
apiVersion: aws.konfig.io/v1alpha1
kind: GlueCatalog
metadata:
  name: example-gluecatalog
  namespace: default
spec:
  allowFullTableExternalDataAccess: 'True'
  catalogProperties:
    customProperties:
      key: value
    dataLakeAccessProperties:
      allowFullTableExternalDataAccess: 'True'
      catalogType: example
      dataLakeAccess: false
      dataTransferRole: example
      kmsKey: example
      managedWorkgroupName: example-gluecatalog
      managedWorkgroupStatus: example
      redshiftDatabaseName: example-gluecatalog
  createDatabaseDefaultPermissions:
  - permissions:
    - example
    principal:
      dataLakePrincipalIdentifier: example
  createTableDefaultPermissions:
  - permissions:
    - example
    principal:
      dataLakePrincipalIdentifier: example
  description: Managed by konfig-konector
  federatedCatalog:
    connectionName: example-gluecatalog
    identifier: example
  name: example-gluecatalog
  overwriteChildResourcePermissionsWithDefault: Accept
  parameters:
    key: value
  providerRef:
    name: example-gluecatalog
    region: us-east-1
  tags:
  - key: example
    value: example
  targetRedshiftCatalog:
    catalogArn: arn:aws:iam::123456789012:role/example-role

Spec

FieldTypeRequiredDescription
allowFullTableExternalDataAccessenum—Allows third-party engines to access data in Amazon S3 locations that are registered with Lake Formation. Allowed: True, False
catalogPropertiesobject—GlueCatalogCatalogProperties is a nested property type of AWS::Glue::Catalog.
catalogProperties.customPropertiesmap<string>—Additional key-value properties for the catalog.
catalogProperties.dataLakeAccessPropertiesobject—GlueCatalogDataLakeAccessProperties is a nested property type of AWS::Glue::Catalog.
catalogProperties.dataLakeAccessProperties.allowFullTableExternalDataAccessenum—Allows third-party engines to access data in Amazon S3 locations that are registered with Lake Formation. Allowed: True, False
catalogProperties.dataLakeAccessProperties.catalogTypestring—Specifies a federated catalog type for the native catalog resource.
catalogProperties.dataLakeAccessProperties.dataLakeAccessboolean—Turns on or off data lake access for Apache Spark applications that access Amazon Redshift databases in the Data Catalog from any non-Redshift engine.
catalogProperties.dataLakeAccessProperties.dataTransferRolestring—A role that will be assumed by Glue for transferring data into/out of the staging bucket during a query.
catalogProperties.dataLakeAccessProperties.kmsKeystring—An encryption key that will be used for the staging bucket that will be created along with the catalog.
catalogProperties.dataLakeAccessProperties.managedWorkgroupNamestring—The name of the managed workgroup associated with the catalog.
catalogProperties.dataLakeAccessProperties.managedWorkgroupStatusstring—The status of the managed workgroup.
catalogProperties.dataLakeAccessProperties.redshiftDatabaseNamestring—The name of the Redshift database.
createDatabaseDefaultPermissionsarray<object>—An array of PrincipalPermissions objects for default database permissions.
createDatabaseDefaultPermissions[].permissionsarray<string>—The permissions that are granted to the principal.
createDatabaseDefaultPermissions[].principalobject—GlueCatalogDataLakePrincipal is a nested property type of AWS::Glue::Catalog.
createDatabaseDefaultPermissions[].principal.dataLakePrincipalIdentifierstring—An identifier for the Lake Formation principal.
createTableDefaultPermissionsarray<object>—An array of PrincipalPermissions objects for default table permissions.
createTableDefaultPermissions[].permissionsarray<string>—The permissions that are granted to the principal.
createTableDefaultPermissions[].principalobject—GlueCatalogDataLakePrincipal is a nested property type of AWS::Glue::Catalog.
createTableDefaultPermissions[].principal.dataLakePrincipalIdentifierstring—An identifier for the Lake Formation principal.
descriptionstring—A description of the catalog.
federatedCatalogobject—GlueCatalogFederatedCatalog is a nested property type of AWS::Glue::Catalog.
federatedCatalog.connectionNamestring—The name of the connection to an external data source.
federatedCatalog.identifierstring—A unique identifier for the federated catalog.
namestringyesThe name of the catalog to create.
overwriteChildResourcePermissionsWithDefaultenum—Specifies whether to overwrite child resource permissions with the default permissions. Allowed: Accept, Deny
parametersmap<string>—A map of key-value pairs that define parameters and properties of the catalog.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
tagsarray<object>—An array of key-value pairs to apply to this resource.
tags[].keystringyes
tags[].valuestringyes
targetRedshiftCatalogobject—GlueCatalogTargetRedshiftCatalog is a nested property type of AWS::Glue::Catalog.
targetRedshiftCatalog.catalogArnstringyesThe Amazon Resource Name (ARN) of the catalog resource.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
catalogIdstring—The ID of the catalog.
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
createTimeinteger—The time at which the catalog was created.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.
resourceArnstring—The Amazon Resource Name (ARN) of the catalog.
updateTimeinteger—The time at which the catalog was last updated.

GlueClassifier # CLOUD CONTROL · AWS::Glue::Classifier

GlueClassifier manages AWS::Glue::Classifier through the AWS Cloud Control API.

kubectl get glueclassifiers · example on GitHub

Example

# GlueClassifier — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueclassifier
apiVersion: aws.konfig.io/v1alpha1
kind: GlueClassifier
metadata:
  name: example-glueclassifier
  namespace: default
spec:
  csvClassifier:
    allowSingleColumn: false
    containsCustomDatatype:
    - example
    containsHeader: example
    customDatatypeConfigured: false
    delimiter: example
    disableValueTrimming: false
    header:
    - example
    name: example-glueclassifier
    quoteSymbol: example
  grokClassifier:
    classification: example
    customPatterns: example
    grokPattern: example
    name: example-glueclassifier
  jsonClassifier:
    jsonPath: example
    name: example-glueclassifier
  providerRef:
    name: example-glueclassifier
    region: us-east-1
  xmlClassifier:
    classification: example
    name: example-glueclassifier
    rowTag: example

Spec

FieldTypeRequiredDescription
csvClassifierobject—A classifier for comma-separated values (CSV).
csvClassifier.allowSingleColumnboolean—Enables the processing of files that contain only one column.
csvClassifier.containsCustomDatatypearray<string>—Indicates whether the CSV file contains custom data types.
csvClassifier.containsHeaderstring—Indicates whether the CSV file contains a header.
csvClassifier.customDatatypeConfiguredboolean—Enables the configuration of custom data types.
csvClassifier.delimiterstring—A custom symbol to denote what separates each column entry in the row.
csvClassifier.disableValueTrimmingboolean—Specifies not to trim values before identifying the type of column values.
csvClassifier.headerarray<string>—A list of strings representing column names.
csvClassifier.namestring—The name of the classifier.
csvClassifier.quoteSymbolstring—A custom symbol to denote what combines content into a single column value.
grokClassifierobject—A classifier that uses grok.
grokClassifier.classificationstringyesAn identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, and so on.
grokClassifier.customPatternsstring—Optional custom grok patterns defined by this classifier.
grokClassifier.grokPatternstringyesThe grok pattern applied to a data store by this classifier.
grokClassifier.namestring—The name of the classifier.
jsonClassifierobject—A classifier for JSON content.
jsonClassifier.jsonPathstringyesA JsonPath string defining the JSON data for the classifier to classify.
jsonClassifier.namestring—The name of the classifier.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
xmlClassifierobject—A classifier for XML content.
xmlClassifier.classificationstringyesAn identifier of the data format that the classifier matches.
xmlClassifier.namestring—The name of the classifier.
xmlClassifier.rowTagstringyesThe XML tag designating the element that contains each record in an XML document being parsed.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
namestring—One of XMLClassifier/Name, GrokClassifier/Name, JsonClassifier/Name or CsvClassifier/Name Read-only: reported by AWS after creation.
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueConnection # GA

GlueConnection is the Schema for managing Glue connections.

kubectl get glueconnections · example on GitHub

Example

# GlueConnection — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueconnection
apiVersion: aws.konfig.io/v1alpha1
kind: GlueConnection
metadata:
  name: example-glueconnection
  namespace: default
spec:
  connectionProperties:
    key: value
  connectionType: JDBC
  description: Managed by konfig-konector
  name: example-glueconnection
  passwordSecretRef:
    key: example
    name: example-glueconnection
  physicalConnectionRequirements:
    availabilityZone: us-east-1a
    securityGroupRefs:
    - name: example-securitygrouprefs
    subnetRef:
      name: example-subnet
  providerRef:
    name: example-glueconnection
    region: us-east-1
  tags:
    environment: dev
    team: platform

Spec

FieldTypeRequiredDescription
connectionPropertiesmap<string>—ConnectionProperties are key-value pairs for the connection (HOST, PORT, USERNAME, JDBC_CONNECTION_URL, ...).
connectionTypeenumyesConnectionType is the type of connection. Allowed: JDBC, KAFKA, MONGODB, NETWORK, MARKETPLACE, CUSTOM
descriptionstring—Description is a description of the connection.
namestringyesName is the name of the connection.
passwordSecretRefobject—PasswordSecretRef references a Kubernetes Secret holding the value of the PASSWORD connection property.
passwordSecretRef.keystringyesKey is the key within the Secret whose value to use.
passwordSecretRef.namestringyesName of the Kubernetes Secret.
physicalConnectionRequirementsobject—PhysicalConnectionRequirements specifies VPC placement for the connection.
physicalConnectionRequirements.availabilityZonestring—AvailabilityZone is the connection's Availability Zone.
physicalConnectionRequirements.securityGroupRefsarray<object>—SecurityGroupRefs reference the security groups the connection uses.
physicalConnectionRequirements.securityGroupRefs[].idstring—ID is a direct AWS Security Group ID (e.g.
physicalConnectionRequirements.securityGroupRefs[].namestring—Name of a SecurityGroup CR in the same namespace.
physicalConnectionRequirements.subnetRefobject—SubnetRef references the subnet the connection uses.
physicalConnectionRequirements.subnetRef.idstring—ID is a direct AWS Subnet ID (e.g.
physicalConnectionRequirements.subnetRef.namestring—Name of a Subnet CR in the same namespace.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
tagsmap<string>—Tags are AWS resource tags to apply at creation.

Status

FieldTypeRequiredDescription
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
connectionNamestring—ConnectionName is the name of the connection in AWS.
lastSyncTimestring—LastSyncTime is when the resource was last successfully reconciled.
observedGenerationinteger—ObservedGeneration is the most recent .metadata.generation reconciled.

GlueCrawler # GA

GlueCrawler is the Schema for managing Glue crawlers.

kubectl get gluecrawlers · example on GitHub

Example

# GlueCrawler — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluecrawler
apiVersion: aws.konfig.io/v1alpha1
kind: GlueCrawler
metadata:
  name: example-gluecrawler
  namespace: default
spec:
  configuration: example
  databaseName: example-gluecrawler
  databaseRef: example
  description: Managed by konfig-konector
  name: example-gluecrawler
  providerRef:
    name: example-gluecrawler
    region: us-east-1
  roleRef:
    name: example-role
  schedule: example
  schemaChangePolicy:
    deleteBehavior: LOG
    updateBehavior: LOG
  tablePrefix: example
  tags:
    environment: dev
    team: platform
  targets:
    jdbcTargets:
    - connectionName: example-gluecrawler
      exclusions:
      - example
      path: example
    s3Targets:
    - exclusions:
      - example
      path: example

Spec

FieldTypeRequiredDescription
configurationstring—Configuration is a versioned JSON configuration string for the crawler.
databaseNamestring—DatabaseName is the Glue database where crawl results are written.
databaseRefstring—DatabaseRef is the name of a GlueDatabase CR in the same namespace.
descriptionstring—Description is a description of the crawler.
namestringyesName is the name of the crawler.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
roleRefobjectyesRoleRef references the IAM role the crawler uses to access resources (either a managed IAMRole CR or a direct ARN).
roleRef.arnstring—ARN is a direct AWS IAM role ARN.
roleRef.namestring—Name of an IAMRole CR in the same namespace.
roleRef.namespacestring—Namespace of the IAMRole CR.
schedulestring—Schedule is a cron expression, e.g.
schemaChangePolicyobject—SchemaChangePolicy sets the crawler's update and deletion behavior.
schemaChangePolicy.deleteBehaviorenum—DeleteBehavior is the behavior when the crawler finds a deleted object. Allowed: LOG, DELETE_FROM_DATABASE, DEPRECATE_IN_DATABASE
schemaChangePolicy.updateBehaviorenum—UpdateBehavior is the behavior when the crawler finds a changed schema. Allowed: LOG, UPDATE_IN_DATABASE
tablePrefixstring—TablePrefix is the prefix for catalog tables the crawler creates.
tagsmap<string>—Tags are AWS resource tags to apply at creation.
targetsobjectyesTargets is the collection of data stores to crawl.
targets.jdbcTargetsarray<object>—JDBCTargets specifies JDBC targets.
targets.jdbcTargets[].connectionNamestringyesConnectionName is the name of the Glue connection to use.
targets.jdbcTargets[].exclusionsarray<string>—Exclusions is a list of glob patterns to exclude from the crawl.
targets.jdbcTargets[].pathstring—Path is the path of the JDBC target.
targets.s3Targetsarray<object>—S3Targets specifies Amazon S3 targets.
targets.s3Targets[].exclusionsarray<string>—Exclusions is a list of glob patterns to exclude from the crawl.
targets.s3Targets[].pathstringyesPath is the path to the Amazon S3 target (e.g.

Status

FieldTypeRequiredDescription
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
crawlerNamestring—CrawlerName is the name of the crawler in AWS.
lastSyncTimestring—LastSyncTime is when the resource was last successfully reconciled.
observedGenerationinteger—ObservedGeneration is the most recent .metadata.generation reconciled.

GlueCustomEntityType # CLOUD CONTROL · AWS::Glue::CustomEntityType

GlueCustomEntityType manages AWS::Glue::CustomEntityType through the AWS Cloud Control API.

kubectl get gluecustomentitytypes · example on GitHub

Example

# GlueCustomEntityType — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluecustomentitytype
apiVersion: aws.konfig.io/v1alpha1
kind: GlueCustomEntityType
metadata:
  name: example-gluecustomentitytype
  namespace: default
spec:
  contextWords:
  - example
  name: example-gluecustomentitytype
  providerRef:
    name: example-gluecustomentitytype
    region: us-east-1
  regexString: example

Spec

FieldTypeRequiredDescription
contextWordsarray<string>—A list of context words.
namestring—The name of the custom entity type.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
regexStringstring—A regular expression string that is used for detecting sensitive data in a custom pattern.
tagsobject—Tags to associate with the custom entity type.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueDataCatalogEncryptionSettings # CLOUD CONTROL · AWS::Glue::DataCatalogEncryptionSettings ⚠ DESTRUCTIVE SCOPE

Caution, destructive scope: account-wide Glue Data Catalog encryption settings. Provided for completeness; do not use from application namespaces. Restrict with AWSProvider.allowedNamespaces and RBAC. GlueDataCatalogEncryptionSettings manages AWS::Glue::DataCatalogEncryptionSettings through the AWS Cloud Control API.

kubectl get gluedatacatalogencryptionsettings · example on GitHub

Example

# GlueDataCatalogEncryptionSettings — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluedatacatalogencryptionsettings
apiVersion: aws.konfig.io/v1alpha1
kind: GlueDataCatalogEncryptionSettings
metadata:
  name: example-gluedatacatalogencryptionsettings
  namespace: default
spec:
  catalogId: example-id
  dataCatalogEncryptionSettings:
    connectionPasswordEncryption:
      kmsKeyId: arn:aws:kms:us-east-1:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab
      returnConnectionPasswordEncrypted: true
    encryptionAtRest:
      catalogEncryptionMode: example
      catalogEncryptionServiceRole: example
      sseAwsKmsKeyId: arn:aws:kms:us-east-1:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab
  providerRef:
    name: example-gluedatacatalogencryptionsettings
    region: us-east-1

Spec

FieldTypeRequiredDescription
catalogIdstringyesThe ID of the Data Catalog in which the settings are created.
dataCatalogEncryptionSettingsobjectyesGlueDataCatalogEncryptionSettingsDataCatalogEncryptionSettings is a nested property type of AWS::Glue::DataCatalogEncryptionSettings.
dataCatalogEncryptionSettings.connectionPasswordEncryptionobject—GlueDataCatalogEncryptionSettingsConnectionPasswordEncryption is a nested property type of AWS::Glue::DataCatalogEncryptionSettings.
dataCatalogEncryptionSettings.connectionPasswordEncryption.kmsKeyIdstring—An AWS KMS key that is used to encrypt the connection password.
dataCatalogEncryptionSettings.connectionPasswordEncryption.returnConnectionPasswordEncryptedboolean—When the ReturnConnectionPasswordEncrypted flag is set to 'true', passwords remain encrypted in the responses of GetConnection and GetConnections.
dataCatalogEncryptionSettings.encryptionAtRestobject—GlueDataCatalogEncryptionSettingsEncryptionAtRest is a nested property type of AWS::Glue::DataCatalogEncryptionSettings.
dataCatalogEncryptionSettings.encryptionAtRest.catalogEncryptionModestring—The encryption-at-rest mode for encrypting Data Catalog data.
dataCatalogEncryptionSettings.encryptionAtRest.catalogEncryptionServiceRolestring—The role that AWS Glue assumes to encrypt and decrypt the Data Catalog objects on the caller's behalf.
dataCatalogEncryptionSettings.encryptionAtRest.sseAwsKmsKeyIdstring—The ID of the AWS KMS key to use for encryption at rest.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueDataQualityRuleset # CLOUD CONTROL · AWS::Glue::DataQualityRuleset

GlueDataQualityRuleset manages AWS::Glue::DataQualityRuleset through the AWS Cloud Control API.

kubectl get gluedataqualityrulesets · example on GitHub

Example

# GlueDataQualityRuleset — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluedataqualityruleset
apiVersion: aws.konfig.io/v1alpha1
kind: GlueDataQualityRuleset
metadata:
  name: example-gluedataqualityruleset
  namespace: default
spec:
  clientToken: example
  description: Managed by konfig-konector
  name: example-gluedataqualityruleset
  providerRef:
    name: example-gluedataqualityruleset
    region: us-east-1
  ruleset: example
  tags:
    environment: dev
    team: platform
  targetTable:
    databaseName: example-gluedataqualityruleset
    tableName: example-gluedataqualityruleset

Spec

FieldTypeRequiredDescription
clientTokenstring—A unique token for idempotency.
descriptionstring—A description of the data quality ruleset.
namestringyesA unique name for the data quality ruleset.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
rulesetstring—A Data Quality Definition Language (DQDL) ruleset.
tagsmap<string>—A map of key-value pairs to apply to this resource.
targetTableobject—An object representing an AWS Glue table.
targetTable.databaseNamestringyesThe name of the database where the AWS Glue table exists.
targetTable.tableNamestringyesThe name of the AWS Glue table.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueDatabase # GA

GlueDatabase is the Schema for managing Glue Data Catalog databases.

kubectl get gluedatabases · example on GitHub

Example

# GlueDatabase — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluedatabase
apiVersion: aws.konfig.io/v1alpha1
kind: GlueDatabase
metadata:
  name: example-gluedatabase
  namespace: default
spec:
  description: Managed by konfig-konector
  locationUri: example
  name: example-gluedatabase
  parameters:
    key: value
  providerRef:
    name: example-gluedatabase
    region: us-east-1
  tags:
    environment: dev
    team: platform

Spec

FieldTypeRequiredDescription
descriptionstring—Description is a description of the database.
locationUristring—LocationURI is the location of the database (for example, an S3 path).
namestringyesName is the name of the database.
parametersmap<string>—Parameters are key-value pairs that define properties of the database.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
tagsmap<string>—Tags are AWS resource tags to apply at creation.

Status

FieldTypeRequiredDescription
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
databaseNamestring—DatabaseName is the name of the database in the Glue Data Catalog.
lastSyncTimestring—LastSyncTime is when the resource was last successfully reconciled.
observedGenerationinteger—ObservedGeneration is the most recent .metadata.generation reconciled.

GlueIdentityCenterConfiguration # CLOUD CONTROL · AWS::Glue::IdentityCenterConfiguration

GlueIdentityCenterConfiguration manages AWS::Glue::IdentityCenterConfiguration through the AWS Cloud Control API.

kubectl get glueidentitycenterconfigurations · example on GitHub

Example

# GlueIdentityCenterConfiguration — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueidentitycenterconfiguration
apiVersion: aws.konfig.io/v1alpha1
kind: GlueIdentityCenterConfiguration
metadata:
  name: example-glueidentitycenterconfiguration
  namespace: default
spec:
  instanceArn: arn:aws:iam::123456789012:role/example-role
  providerRef:
    name: example-glueidentitycenterconfiguration
    region: us-east-1
  scopes:
  - example
  userBackgroundSessionsEnabled: true

Spec

FieldTypeRequiredDescription
instanceArnstringyesThe IAM identity center instance arn Create-only in AWS: changing it requires replacing the resource.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
scopesarray<string>—The downstream scopes that Glue identity center configuration can access
userBackgroundSessionsEnabledboolean—Enable or disable user background sessions for Glue Identity Center

Status

FieldTypeRequiredDescription
accountIdstring—The identifier for the specified AWS account.
applicationArnstring—The Glue IAM identity center application arn Read-only: reported by AWS after creation.
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueIntegration # CLOUD CONTROL · AWS::Glue::Integration

GlueIntegration manages AWS::Glue::Integration through the AWS Cloud Control API.

kubectl get glueintegrations · example on GitHub

Example

# GlueIntegration — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueintegration
apiVersion: aws.konfig.io/v1alpha1
kind: GlueIntegration
metadata:
  name: example-glueintegration
  namespace: default
spec:
  additionalEncryptionContext:
    key: value
  dataFilter: example
  description: Managed by konfig-konector
  integrationConfig:
    continuousSync: false
    refreshInterval: example
    sourceProperties:
      key: value
  integrationName: example-glueintegration
  kmsKeyId: arn:aws:kms:us-east-1:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab
  providerRef:
    name: example-glueintegration
    region: us-east-1
  sourceArn: arn:aws:iam::123456789012:role/example-role
  tags:
  - key: example
    value: example
  targetArn: arn:aws:iam::123456789012:role/example-role

Spec

FieldTypeRequiredDescription
additionalEncryptionContextmap<string>—Create-only in AWS: changing it requires replacing the resource.
dataFilterstring—
descriptionstring—
integrationConfigobject—Create-only in AWS: changing it requires replacing the resource.
integrationConfig.continuousSyncboolean—Enables continuous synchronization for on-demand data extractions.
integrationConfig.refreshIntervalstring—Specifies the frequency at which CDC (Change Data Capture) pulls or incremental loads should occur.
integrationConfig.sourcePropertiesmap<string>—
integrationNamestringyesThe name of the integration.
kmsKeyIdstring—An KMS key identifier for the key to use to encrypt the integration.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
sourceArnstringyesThe Amazon Resource Name (ARN) of the database to use as the source for replication Create-only in AWS: changing it requires replacing the resource.
tagsarray<object>—An array of key-value pairs to apply to this resource.
tags[].keystringyes
tags[].valuestringyes
targetArnstringyesThe Amazon Resource Name (ARN) of the Glue data warehouse to use as the target for replication Create-only in AWS: changing it requires replacing the resource.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
createTimestring—The time (UTC) when the integration was created.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
integrationArnstring—The Amazon Resource Name (ARN) of the integration.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.
statusstring—The status of the integration.

GlueIntegrationResourceProperty # CLOUD CONTROL · AWS::Glue::IntegrationResourceProperty

GlueIntegrationResourceProperty manages AWS::Glue::IntegrationResourceProperty through the AWS Cloud Control API.

kubectl get glueintegrationresourceproperties · example on GitHub

Example

# GlueIntegrationResourceProperty — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueintegrationresourceproperty
apiVersion: aws.konfig.io/v1alpha1
kind: GlueIntegrationResourceProperty
metadata:
  name: example-glueintegrationresourceproperty
  namespace: default
spec:
  providerRef:
    name: example-glueintegrationresourceproperty
    region: us-east-1
  resourceArn: arn:aws:iam::123456789012:role/example-role
  sourceProcessingProperties:
    roleArn: arn:aws:iam::123456789012:role/example-role
  tags:
  - key: example
    value: example
  targetProcessingProperties:
    connectionName: example-glueintegrationresourceproperty
    eventBusArn: arn:aws:iam::123456789012:role/example-role
    kmsArn: arn:aws:iam::123456789012:role/example-role
    roleArn: arn:aws:iam::123456789012:role/example-role

Spec

FieldTypeRequiredDescription
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
resourceArnstringyesThe connection ARN of the source, or the database ARN of the target.
sourceProcessingPropertiesobject—The resource properties associated with the integration source.
sourceProcessingProperties.roleArnstringyesThe IAM role to access the Glue connection.
tagsarray<object>—An array of key-value pairs to apply to this resource.
tags[].keystringyes
tags[].valuestringyes
targetProcessingPropertiesobject—The resource properties associated with the integration target.
targetProcessingProperties.connectionNamestring—The Glue network connection to configure the Glue job running in the customer VPC.
targetProcessingProperties.eventBusArnstring—The ARN of an Eventbridge event bus to receive the integration status notification.
targetProcessingProperties.kmsArnstring—The ARN of the KMS key used for encryption.
targetProcessingProperties.roleArnstringyesThe IAM role to access the Glue database.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.
resourcePropertyArnstring—The integration resource property ARN.

GlueJob # GA

GlueJob is the Schema for managing Glue jobs.

kubectl get gluejobs · example on GitHub

Example

# GlueJob — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluejob
apiVersion: aws.konfig.io/v1alpha1
kind: GlueJob
metadata:
  name: example-gluejob
  namespace: default
spec:
  command:
    name: example-gluejob
    pythonVersion: '1.0'
    scriptLocation: example
  defaultArguments:
    key: value
  description: Managed by konfig-konector
  glueVersion: '1.0'
  maxRetries: 1
  name: example-gluejob
  numberOfWorkers: 1
  providerRef:
    name: example-gluejob
    region: us-east-1
  roleRef:
    name: example-role
  tags:
    environment: dev
    team: platform
  timeout: 1
  workerType: Standard

Spec

FieldTypeRequiredDescription
commandobjectyesCommand specifies the code the job runs.
command.namestringyesName is the job command name: glueetl for Spark ETL, pythonshell for Python shell, gluestreaming for streaming ETL.
command.pythonVersionstring—PythonVersion is the Python version for pythonshell jobs (2 or 3).
command.scriptLocationstringyesScriptLocation is the S3 path of the script that runs the job.
defaultArgumentsmap<string>—DefaultArguments are default arguments for job runs.
descriptionstring—Description is a description of the job.
glueVersionstring—GlueVersion determines the Glue runtime versions (e.g.
maxRetriesinteger—MaxRetries is the maximum number of retries for a failing job run.
namestringyesName is the name of the job.
numberOfWorkersinteger—NumberOfWorkers is the number of workers allocated when the job runs.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
roleRefobjectyesRoleRef references the IAM role the job uses (either a managed IAMRole CR or a direct ARN).
roleRef.arnstring—ARN is a direct AWS IAM role ARN.
roleRef.namestring—Name of an IAMRole CR in the same namespace.
roleRef.namespacestring—Namespace of the IAMRole CR.
tagsmap<string>—Tags are AWS resource tags to apply at creation.
timeoutinteger—Timeout is the job run timeout in minutes.
workerTypeenum—WorkerType is the type of predefined worker allocated when the job runs. Allowed: Standard, G.1X, G.2X, G.025X, G.4X, G.8X, Z.2X

Status

FieldTypeRequiredDescription
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
jobNamestring—JobName is the name of the job in AWS.
lastSyncTimestring—LastSyncTime is when the resource was last successfully reconciled.
observedGenerationinteger—ObservedGeneration is the most recent .metadata.generation reconciled.

GlueMLTransform # CLOUD CONTROL · AWS::Glue::MLTransform

GlueMLTransform manages AWS::Glue::MLTransform through the AWS Cloud Control API.

kubectl get gluemltransforms · example on GitHub

Example

# GlueMLTransform — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluemltransform
apiVersion: aws.konfig.io/v1alpha1
kind: GlueMLTransform
metadata:
  name: example-gluemltransform
  namespace: default
spec:
  description: Managed by konfig-konector
  glueVersion: '1.0'
  inputRecordTables:
    glueTables:
    - connectionName: example-gluemltransform
      databaseName: example-gluemltransform
      tableName: example-gluemltransform
  maxCapacity: 2
  maxRetries: 1
  name: example-gluemltransform
  numberOfWorkers: 1
  providerRef:
    name: example-gluemltransform
    region: us-east-1
  role: example
  tags:
    environment: dev
    team: platform
  timeout: 300
  transformEncryption:
    mlUserDataEncryption:
      kmsKeyId: arn:aws:kms:us-east-1:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab
      mlUserDataEncryptionMode: example
    taskRunSecurityConfigurationName: example-gluemltransform
  transformParameters:
    findMatchesParameters:
      accuracyCostTradeoff: 1
      enforceProvidedLabels: false
      precisionRecallTradeoff: 1
      primaryKeyColumnName: example-gluemltransform
    transformType: example
  workerType: example

Spec

FieldTypeRequiredDescription
descriptionstring—A user-defined, long-form description text for the machine learning transform.
glueVersionstring—The version of AWS Glue this machine learning transform is compatible with.
inputRecordTablesobjectyesA list of AWS Glue table definitions used by the transform.
inputRecordTables.glueTablesarray<object>—The database and table in the AWS Glue Data Catalog that is used for input or output data.
inputRecordTables.glueTables[].catalogIdstring—A unique identifier for the AWS Glue Data Catalog.
inputRecordTables.glueTables[].connectionNamestring—The name of the connection to the AWS Glue Data Catalog.
inputRecordTables.glueTables[].databaseNamestringyesA database name in the AWS Glue Data Catalog.
inputRecordTables.glueTables[].tableNamestringyesA table name in the AWS Glue Data Catalog.
maxCapacitynumber—The number of AWS Glue DPUs allocated to task runs for this transform.
maxRetriesinteger—The maximum number of times to retry after an MLTaskRun fails.
namestring—A user-defined name for the machine learning transform.
numberOfWorkersinteger—The number of workers of a defined workerType that are allocated when a task runs.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
rolestringyesThe name or ARN of the IAM role with the required permissions.
tagsmap<string>—The tags to use with this machine learning transform.
timeoutinteger—The timeout in minutes of the machine learning transform.
transformEncryptionobject—The encryption-at-rest settings of the transform.
transformEncryption.mlUserDataEncryptionobject—GlueMLTransformMLUserDataEncryption is a nested property type of AWS::Glue::MLTransform.
transformEncryption.mlUserDataEncryption.kmsKeyIdstring—The ID for the customer-provided KMS key.
transformEncryption.mlUserDataEncryption.mlUserDataEncryptionModestringyesThe encryption mode applied to user data.
transformEncryption.taskRunSecurityConfigurationNamestring—The name of the security configuration.
transformParametersobjectyesThe algorithm-specific parameters that are associated with the machine learning transform.
transformParameters.findMatchesParametersobject—GlueMLTransformFindMatchesParameters is a nested property type of AWS::Glue::MLTransform.
transformParameters.findMatchesParameters.accuracyCostTradeoffnumber—The value for accuracy and cost tradeoff.
transformParameters.findMatchesParameters.enforceProvidedLabelsboolean—If true, forces the output to match the provided labels.
transformParameters.findMatchesParameters.precisionRecallTradeoffnumber—The value for precision and recall tradeoff.
transformParameters.findMatchesParameters.primaryKeyColumnNamestringyesThe name of a column that uniquely identifies rows in the source table.
transformParameters.transformTypestringyesThe type of machine learning transform.
workerTypestring—The type of predefined worker that is allocated when a task runs.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.
transformIdstring—The unique identifier for the transform.

GlueRegistry # CLOUD CONTROL · AWS::Glue::Registry

GlueRegistry manages AWS::Glue::Registry through the AWS Cloud Control API.

kubectl get glueregistries · example on GitHub

Example

# GlueRegistry — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueregistry
apiVersion: aws.konfig.io/v1alpha1
kind: GlueRegistry
metadata:
  name: example-glueregistry
  namespace: default
spec:
  description: Managed by konfig-konector
  name: example-glueregistry
  providerRef:
    name: example-glueregistry
    region: us-east-1
  tags:
  - key: example
    value: example

Spec

FieldTypeRequiredDescription
descriptionstring—A description of the registry.
namestringyesName of the registry to be created of max length of 255, and may only contain letters, numbers, hyphen, underscore, dollar sign, or hash mark.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
tagsarray<object>—List of tags to tag the Registry
tags[].keystringyes
tags[].valuestringyes

Status

FieldTypeRequiredDescription
arnstring—Amazon Resource Name for the created Registry.
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueSchema # CLOUD CONTROL · AWS::Glue::Schema

GlueSchema manages AWS::Glue::Schema through the AWS Cloud Control API.

kubectl get glueschemas · example on GitHub

Example

# GlueSchema — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueschema
apiVersion: aws.konfig.io/v1alpha1
kind: GlueSchema
metadata:
  name: example-glueschema
  namespace: default
spec:
  checkpointVersion:
    isLatest: false
    versionNumber: 1
  compatibility: NONE
  dataFormat: AVRO
  description: Managed by konfig-konector
  name: example-glueschema
  providerRef:
    name: example-glueschema
    region: us-east-1
  registry:
    name: example-registry
  schemaDefinition: example
  tags:
  - key: example
    value: example

Spec

FieldTypeRequiredDescription
checkpointVersionobject—GlueSchemaSchemaVersion is a nested property type of AWS::Glue::Schema.
checkpointVersion.isLatestboolean—Indicates if the latest version needs to be updated.
checkpointVersion.versionNumberinteger—Indicates the version number in the schema to update.
compatibilityenumyesCompatibility setting for the schema. Allowed: NONE, DISABLED, BACKWARD, BACKWARD_ALL, FORWARD, FORWARD_ALL, FULL, FULL_ALL
dataFormatenumyesData format name to use for the schema. Allowed: AVRO, JSON, PROTOBUF
descriptionstring—A description of the schema.
namestringyesName of the schema.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
registryobject—Create-only in AWS: changing it requires replacing the resource.
registry.arnstring—Amazon Resource Name for the Registry.
registry.namestring—Name of the registry in which the schema will be created.
schemaDefinitionstring—Definition for the initial schema version in plain-text.
tagsarray<object>—List of tags to tag the schema
tags[].keystringyes
tags[].valuestringyes

Status

FieldTypeRequiredDescription
arnstring—Amazon Resource Name for the Schema.
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
initialSchemaVersionIdstring—Represents the version ID associated with the initial schema version.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueSchemaVersion # CLOUD CONTROL · AWS::Glue::SchemaVersion

GlueSchemaVersion manages AWS::Glue::SchemaVersion through the AWS Cloud Control API.

kubectl get glueschemaversions · example on GitHub

Example

# GlueSchemaVersion — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueschemaversion
apiVersion: aws.konfig.io/v1alpha1
kind: GlueSchemaVersion
metadata:
  name: example-glueschemaversion
  namespace: default
spec:
  providerRef:
    name: example-glueschemaversion
    region: us-east-1
  schema:
    registryName: example-glueschemaversion
    schemaArn: arn:aws:iam::123456789012:role/example-role
    schemaName: example-glueschemaversion
  schemaDefinition: example

Spec

FieldTypeRequiredDescription
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
schemaobjectyesCreate-only in AWS: changing it requires replacing the resource.
schema.registryNamestring—Name of the registry to identify where the Schema is located.
schema.schemaArnstring—Amazon Resource Name for the Schema.
schema.schemaNamestring—Name of the schema.
schemaDefinitionstringyesComplete definition of the schema in plain-text.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.
versionIdstring—Represents the version ID associated with the schema version.

GlueSchemaVersionMetadata # CLOUD CONTROL · AWS::Glue::SchemaVersionMetadata

GlueSchemaVersionMetadata manages AWS::Glue::SchemaVersionMetadata through the AWS Cloud Control API.

kubectl get glueschemaversionmetadata · example on GitHub

Example

# GlueSchemaVersionMetadata — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueschemaversionmetadata
apiVersion: aws.konfig.io/v1alpha1
kind: GlueSchemaVersionMetadata
metadata:
  name: example-glueschemaversionmetadata
  namespace: default
spec:
  key: example
  providerRef:
    name: example-glueschemaversionmetadata
    region: us-east-1
  schemaVersionId: example-id
  value: example

Spec

FieldTypeRequiredDescription
keystringyesMetadata key Create-only in AWS: changing it requires replacing the resource.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
schemaVersionIdstringyesRepresents the version ID associated with the schema version.
valuestringyesMetadata value Create-only in AWS: changing it requires replacing the resource.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueSecurityConfiguration # CLOUD CONTROL · AWS::Glue::SecurityConfiguration

GlueSecurityConfiguration manages AWS::Glue::SecurityConfiguration through the AWS Cloud Control API.

kubectl get gluesecurityconfigurations · example on GitHub

Example

# GlueSecurityConfiguration — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluesecurityconfiguration
apiVersion: aws.konfig.io/v1alpha1
kind: GlueSecurityConfiguration
metadata:
  name: example-gluesecurityconfiguration
  namespace: default
spec:
  encryptionConfiguration:
    cloudWatchEncryption:
      cloudWatchEncryptionMode: example
      kmsKeyArn: arn:aws:iam::123456789012:role/example-role
    jobBookmarksEncryption:
      jobBookmarksEncryptionMode: example
      kmsKeyArn: arn:aws:iam::123456789012:role/example-role
    s3Encryptions:
    - kmsKeyArn: arn:aws:iam::123456789012:role/example-role
      s3EncryptionMode: example
  name: example-gluesecurityconfiguration
  providerRef:
    name: example-gluesecurityconfiguration
    region: us-east-1

Spec

FieldTypeRequiredDescription
encryptionConfigurationobjectyesGlueSecurityConfigurationEncryptionConfiguration is a nested property type of AWS::Glue::SecurityConfiguration.
encryptionConfiguration.cloudWatchEncryptionobject—GlueSecurityConfigurationCloudWatchEncryption is a nested property type of AWS::Glue::SecurityConfiguration.
encryptionConfiguration.cloudWatchEncryption.cloudWatchEncryptionModestring—
encryptionConfiguration.cloudWatchEncryption.kmsKeyArnstring—
encryptionConfiguration.jobBookmarksEncryptionobject—GlueSecurityConfigurationJobBookmarksEncryption is a nested property type of AWS::Glue::SecurityConfiguration.
encryptionConfiguration.jobBookmarksEncryption.jobBookmarksEncryptionModestring—
encryptionConfiguration.jobBookmarksEncryption.kmsKeyArnstring—
encryptionConfiguration.s3Encryptionsarray<object>—The encryption configuration for Amazon Simple Storage Service (Amazon S3) data.
encryptionConfiguration.s3Encryptions[].kmsKeyArnstring—
encryptionConfiguration.s3Encryptions[].s3EncryptionModestring—
namestringyesThe name for the security configuration.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueSession # CLOUD CONTROL · AWS::Glue::Session

GlueSession manages AWS::Glue::Session through the AWS Cloud Control API.

kubectl get gluesessions · example on GitHub

Example

# GlueSession — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluesession
apiVersion: aws.konfig.io/v1alpha1
kind: GlueSession
metadata:
  name: example-gluesession
  namespace: default
spec:
  command:
    name: example-gluesession
    pythonVersion: '1.0'
  connections:
    connections:
    - example
  defaultArguments:
    key: value
  description: Managed by konfig-konector
  glueVersion: '1.0'
  id: example-id
  idleTimeout: 300
  maxCapacity: 2
  numberOfWorkers: 1
  providerRef:
    name: example-gluesession
    region: us-east-1
  requestOrigin: example
  role: example-xxxxxxxxxxxx
  securityConfiguration: example
  tags:
  - key: example
    value: example
  timeout: 1
  workerType: Standard

Spec

FieldTypeRequiredDescription
commandobjectyesCreate-only in AWS: changing it requires replacing the resource.
command.namestring—Specifies the name of the SessionCommand.
command.pythonVersionstring—Specifies the Python version.
connectionsobject—Create-only in AWS: changing it requires replacing the resource.
connections.connectionsarray<string>—A list of connection names used by the session.
defaultArgumentsmap<string>—A map array of key-value pairs.
descriptionstring—The description of the session.
glueVersionstring—The Glue version determines the versions of Apache Spark and Python that Glue supports.
idstringyesThe ID of the session.
idleTimeoutinteger—The number of minutes when idle before session times out.
maxCapacitynumber—The number of Glue data processing units (DPUs) that can be allocated when the job runs.
numberOfWorkersinteger—The number of workers of a defined WorkerType to use for the session.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
requestOriginstring—The origin of the request.
rolestringyesThe IAM Role ARN.
securityConfigurationstring—The name of the SecurityConfiguration structure to be used with the session.
tagsarray<object>—The tags belonging to the session.
tags[].keystringyes
tags[].valuestringyes
timeoutinteger—The number of minutes before session times out.
workerTypeenum—The type of predefined worker that is allocated when a session runs. Allowed: Standard, G.1X, G.2X, G.025X, G.4X, G.8X, Z.2X

Status

FieldTypeRequiredDescription
arnstring—The Amazon Resource Name (ARN) of the session.
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
createdOnstring—The time and date when the session was created.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
progressnumber—The code execution progress of the session.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.
statusenum—The session status. Allowed: PROVISIONING, READY, FAILED, TIMEOUT, STOPPING, STOPPED

GlueTableOptimizer # CLOUD CONTROL · AWS::Glue::TableOptimizer

GlueTableOptimizer manages AWS::Glue::TableOptimizer through the AWS Cloud Control API.

kubectl get gluetableoptimizers · example on GitHub

Example

# GlueTableOptimizer — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluetableoptimizer
apiVersion: aws.konfig.io/v1alpha1
kind: GlueTableOptimizer
metadata:
  name: example-gluetableoptimizer
  namespace: default
spec:
  catalogId: example-id
  databaseName: example-gluetableoptimizer
  providerRef:
    name: example-gluetableoptimizer
    region: us-east-1
  tableName: example-gluetableoptimizer
  tableOptimizerConfiguration:
    compactionConfiguration:
      icebergConfiguration:
        deleteFileThreshold: 1
        minInputFiles: 1
        strategy: example
    enabled: true
    orphanFileDeletionConfiguration:
      icebergConfiguration:
        location: example
        orphanFileRetentionPeriodInDays: 7
    retentionConfiguration:
      icebergConfiguration:
        cleanExpiredFiles: false
        numberOfSnapshotsToRetain: 1
        snapshotRetentionPeriodInDays: 7
    roleArn: arn:aws:iam::123456789012:role/example-role
    vpcConfiguration:
      glueConnectionName: example-gluetableoptimizer
  type: example

Spec

FieldTypeRequiredDescription
catalogIdstringyesThe catalog ID of the table Create-only in AWS: changing it requires replacing the resource.
databaseNamestringyesThe name of the database.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
tableNamestringyesThe table name.
tableOptimizerConfigurationobjectyesSpecifies configuration details of a table optimizer.
tableOptimizerConfiguration.compactionConfigurationobject—The configuration for a compaction optimizer.
tableOptimizerConfiguration.compactionConfiguration.icebergConfigurationobject—The configuration for an Iceberg compaction optimizer.
tableOptimizerConfiguration.compactionConfiguration.icebergConfiguration.deleteFileThresholdinteger—The minimum number of deletes in a data file to make it eligible for compaction.
tableOptimizerConfiguration.compactionConfiguration.icebergConfiguration.minInputFilesinteger—The minimum number of input files before compaction is triggered.
tableOptimizerConfiguration.compactionConfiguration.icebergConfiguration.strategystring—The compaction strategy to use.
tableOptimizerConfiguration.enabledbooleanyesWhether the table optimization is enabled.
tableOptimizerConfiguration.orphanFileDeletionConfigurationobject—OrphanFileDeletionConfiguration is a property that can be included within the TableOptimizer resource.
tableOptimizerConfiguration.orphanFileDeletionConfiguration.icebergConfigurationobject—The IcebergConfiguration property helps optimize your Iceberg tables in AWS Glue by allowing you to specify format-specific settings that control how data is stored, compressed, and managed.
tableOptimizerConfiguration.orphanFileDeletionConfiguration.icebergConfiguration.locationstring—Specifies a directory in which to look for orphan files (defaults to the table's location).
tableOptimizerConfiguration.orphanFileDeletionConfiguration.icebergConfiguration.orphanFileRetentionPeriodInDaysinteger—The specific number of days you want to keep the orphan files.
tableOptimizerConfiguration.retentionConfigurationobject—The configuration for a snapshot retention optimizer for Apache Iceberg tables.
tableOptimizerConfiguration.retentionConfiguration.icebergConfigurationobject—The configuration for an Iceberg snapshot retention optimizer.
tableOptimizerConfiguration.retentionConfiguration.icebergConfiguration.cleanExpiredFilesboolean—
tableOptimizerConfiguration.retentionConfiguration.icebergConfiguration.numberOfSnapshotsToRetaininteger—
tableOptimizerConfiguration.retentionConfiguration.icebergConfiguration.snapshotRetentionPeriodInDaysinteger—
tableOptimizerConfiguration.roleArnstringyesA role passed by the caller which gives the service permission to update the resources associated with the optimizer on the caller's behalf.
tableOptimizerConfiguration.vpcConfigurationobject—An object that describes the VPC configuration for a table optimizer.
tableOptimizerConfiguration.vpcConfiguration.glueConnectionNamestring—The name of the AWS Glue connection used for the VPC for the table optimizer.
typestringyesThe type of table optimizer.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueTrigger # GA

GlueTrigger is the Schema for managing Glue triggers.

kubectl get gluetriggers · example on GitHub

Example

# GlueTrigger — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#gluetrigger
apiVersion: aws.konfig.io/v1alpha1
kind: GlueTrigger
metadata:
  name: example-gluetrigger
  namespace: default
spec:
  actions:
  - arguments:
      key: value
    jobName: example-gluetrigger
  description: Managed by konfig-konector
  name: example-gluetrigger
  predicate:
    conditions:
    - crawlerName: example-gluetrigger
      jobName: example-gluetrigger
      logicalOperator: EQUALS
      state: example
    logical: AND
  providerRef:
    name: example-gluetrigger
    region: us-east-1
  schedule: example
  startOnCreation: false
  tags:
    environment: dev
    team: platform
  type: SCHEDULED

Spec

FieldTypeRequiredDescription
actionsarray<object>yesActions are initiated when the trigger fires.
actions[].argumentsmap<string>—Arguments are job arguments that replace the job's default arguments for this run.
actions[].jobNamestringyesJobName is the name of the Glue job to run.
descriptionstring—Description is a description of the trigger.
namestringyesName is the name of the trigger.
predicateobject—Predicate defines when a CONDITIONAL trigger fires.
predicate.conditionsarray<object>—Conditions determine when the trigger fires.
predicate.conditions[].crawlerNamestring—CrawlerName is the crawler this condition applies to.
predicate.conditions[].jobNamestring—JobName is the job whose runs this condition applies to.
predicate.conditions[].logicalOperatorenum—LogicalOperator is the comparison operator; only EQUALS is supported. Allowed: EQUALS
predicate.conditions[].statestring—State is the job/crawler state the trigger listens for (SUCCEEDED, STOPPED, FAILED, TIMEOUT for jobs).
predicate.logicalenum—Logical is how multiple conditions combine: AND or ANY. Allowed: AND, ANY
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
schedulestring—Schedule is a cron expression for SCHEDULED triggers, e.g.
startOnCreationboolean—StartOnCreation activates SCHEDULED/CONDITIONAL triggers when created.
tagsmap<string>—Tags are AWS resource tags to apply at creation.
typeenumyesType is the trigger type. Allowed: SCHEDULED, CONDITIONAL, ON_DEMAND

Status

FieldTypeRequiredDescription
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
lastSyncTimestring—LastSyncTime is when the resource was last successfully reconciled.
observedGenerationinteger—ObservedGeneration is the most recent .metadata.generation reconciled.
statestring—State is the current trigger state reported by AWS.
triggerNamestring—TriggerName is the name of the trigger in AWS.

GlueUsageProfile # CLOUD CONTROL · AWS::Glue::UsageProfile

GlueUsageProfile manages AWS::Glue::UsageProfile through the AWS Cloud Control API.

kubectl get glueusageprofiles · example on GitHub

Example

# GlueUsageProfile — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueusageprofile
apiVersion: aws.konfig.io/v1alpha1
kind: GlueUsageProfile
metadata:
  name: example-glueusageprofile
  namespace: default
spec:
  description: Managed by konfig-konector
  name: example-glueusageprofile
  providerRef:
    name: example-glueusageprofile
    region: us-east-1
  tags:
  - key: example
    value: example

Spec

FieldTypeRequiredDescription
configurationobject—UsageProfile configuration for supported service ex: (Jobs, Sessions).
configuration.jobConfigurationobject—
configuration.sessionConfigurationobject—
descriptionstring—The description of the UsageProfile.
namestringyesThe name of the UsageProfile.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
tagsarray<object>—The tags to be applied to this UsageProfiles.
tags[].keystringyes
tags[].valuestringyes

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
createdOnstring—Creation time.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueUserDefinedFunction # CLOUD CONTROL · AWS::Glue::UserDefinedFunction

GlueUserDefinedFunction manages AWS::Glue::UserDefinedFunction through the AWS Cloud Control API.

kubectl get glueuserdefinedfunctions · example on GitHub

Example

# GlueUserDefinedFunction — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueuserdefinedfunction
apiVersion: aws.konfig.io/v1alpha1
kind: GlueUserDefinedFunction
metadata:
  name: example-glueuserdefinedfunction
  namespace: default
spec:
  className: example-glueuserdefinedfunction
  databaseName: example-glueuserdefinedfunction
  functionName: example-glueuserdefinedfunction
  functionType: REGULAR_FUNCTION
  ownerName: example-glueuserdefinedfunction
  ownerType: USER
  providerRef:
    name: example-glueuserdefinedfunction
    region: us-east-1
  resourceUris:
  - resourceType: JAR
    uri: example

Spec

FieldTypeRequiredDescription
classNamestring—The Java class that contains the function code.
databaseNamestringyesThe name of the catalog database in which the function is located.
functionNamestringyesThe name of the function.
functionTypeenum—The type of the function. Allowed: REGULAR_FUNCTION, AGGREGATE_FUNCTION, STORED_PROCEDURE
ownerNamestring—The owner of the function.
ownerTypeenum—The owner type. Allowed: USER, ROLE, GROUP
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
resourceUrisarray<object>—The resource URIs for the function.
resourceUris[].resourceTypeenum—The type of the resource. Allowed: JAR, FILE, ARCHIVE
resourceUris[].uristring—The URI for accessing the resource.

Status

FieldTypeRequiredDescription
arnstring—The Amazon Resource Name (ARN) of the user-defined function.
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.

GlueWorkflow # CLOUD CONTROL · AWS::Glue::Workflow

GlueWorkflow manages AWS::Glue::Workflow through the AWS Cloud Control API.

kubectl get glueworkflows · example on GitHub

Example

# GlueWorkflow — Glue
# Full-options example generated from the CRD schema. Optional fields are
# included with sample values; delete what you don't need. Field reference:
# https://konfig-konector.io/docs/glue.html#glueworkflow
apiVersion: aws.konfig.io/v1alpha1
kind: GlueWorkflow
metadata:
  name: example-glueworkflow
  namespace: default
spec:
  description: Managed by konfig-konector
  maxConcurrentRuns: 1
  name: example-glueworkflow
  providerRef:
    name: example-glueworkflow
    region: us-east-1

Spec

FieldTypeRequiredDescription
defaultRunPropertiesobject—A collection of properties to be used as part of each execution of the workflow
descriptionstring—A description of the workflow
maxConcurrentRunsinteger—You can use this parameter to prevent unwanted multiple updates to data, to control costs, or in some cases, to prevent exceeding the maximum number of concurrent runs of any of the component jobs.
namestring—The name of the workflow representing the flow Create-only in AWS: changing it requires replacing the resource.
providerRefobject—ProviderRef selects the AWSProvider (account/region) this resource is reconciled against.
providerRef.namestringyesName of a cluster-scoped AWSProvider.
providerRef.regionstring—Region overrides the provider's region for this resource only.
tagsobject—The tags to use with this workflow.

Status

FieldTypeRequiredDescription
attemptinteger—Attempt counts failed create/delete requests; it salts the Cloud Control idempotency token so a retry is not served the cached failure.
awsProviderobject—AWSProvider confirms the account and region this resource was reconciled against.
awsProvider.accountIdstring—AccountID the resource lives in.
awsProvider.namestring—Name of the AWSProvider used; empty when the operator's own credentials were used.
awsProvider.regionstring—Region the resource lives in (empty for global services).
conditionsarray<object>—Conditions describe the current state of the resource.
conditions[].lastTransitionTimestringyeslastTransitionTime is the last time the condition transitioned from one status to another.
conditions[].messagestringyesmessage is a human readable message indicating details about the transition.
conditions[].observedGenerationinteger—observedGeneration represents the .metadata.generation that the condition was set based upon.
conditions[].reasonstringyesreason contains a programmatic identifier indicating the reason for the condition's last transition.
conditions[].statusenumyesstatus of the condition, one of True, False, Unknown. Allowed: True, False, Unknown
conditions[].typestringyestype of condition in CamelCase or in foo.example.com/CamelCase.
identifierstring—Identifier is the Cloud Control primary identifier of the resource.
lastSyncTimestring—
observedGenerationinteger—
operationstring—Operation is the in-flight operation (CREATE, UPDATE, DELETE).
operationStatusstring—OperationStatus is the last reported operation status.
requestTokenstring—RequestToken tracks an in-flight asynchronous operation.