Generate Crossplane Providers from any Terraform Provider

Overview

Terrajet - Generate Crossplane Providers from any Terraform Provider

Terrajet is a code generator framework that allows developers to build code generation pipelines that can generate Crossplane controllers. Developers can start building their code generation pipeline targeting specific Terraform Providers by importing Terrajet and wiring all generators together, customizing the whole pipeline in the process.

See design document for more details.

Feel free to test the following Crossplane providers built using Terrajet:

NOTE: Terrajet is in its very early stages. We expect many breaking changes in the coming weeks. Relying on it for production usage is not recommended yet.

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, please open an issue.

Contact

Please use the following to reach members of the community:

Governance and Owners

terrajet is run according to the same Governance and Ownership structure as the core Crossplane project.

Prior Art

There are many projects in infrastructure space that builds on top of Terraform. Each of the projects have their own limitations, additional features and different license restrictions.

Code of Conduct

terrajet adheres to the same Code of Conduct as the core Crossplane project.

Licensing

terrajet is under the Apache 2.0 license.

FOSSA Status

Comments
  • Issues of generating provider-jet-alibaba

    Issues of generating provider-jet-alibaba

    What happened?

    I tried to generate a Terraform provider in the repo https://github.com/zzxwill/provider-jet-alibaba and it failed.

    ➜  /Users/zhouzhengi/Programming/golang/src/github.com/zzxwill/provider-jet-alibaba git:(main) ✗ make generate
    16:42:06 [ .. ] verify dependencies have expected content
    all modules verified
    16:42:21 [ OK ] go modules dependencies verified
    16:42:22 [ .. ] go generate darwin_amd64
    go: downloading github.com/turkenh/terraform-plugin-sdk v1.17.2-patch1
    # github.com/hashicorp/terraform-plugin-sdk/internal/configs/configload
    ../.work/pkg/pkg/mod/github.com/turkenh/[email protected]/internal/configs/configload/loader_snapshot.go:52:29: cannot use fs (type snapshotFS) as type afero.Fs in argument to configs.NewParser:
    	snapshotFS does not implement afero.Fs (missing Chown method)
    ../.work/pkg/pkg/mod/github.com/turkenh/[email protected]/internal/configs/configload/loader_snapshot.go:57:28: cannot use fs (type snapshotFS) as type afero.Fs in field value:
    	snapshotFS does not implement afero.Fs (missing Chown method)
    ../.work/pkg/pkg/mod/github.com/turkenh/[email protected]/internal/configs/configload/loader_snapshot.go:200:5: cannot use snapshotFS{} (type snapshotFS) as type afero.Fs in assignment:
    	snapshotFS does not implement afero.Fs (missing Chown method)
    apis/generate.go:32: running "go": exit status 2
    16:42:39 [FAIL]
    make[1]: *** [go.generate] Error 1
    make: *** [generate] Error 2
    

    How can we reproduce it?

    1. Clone https://github.com/zzxwill/provider-jet-alibaba.git
    2. run make generate
    bug 
    opened by zzxwill 20
  • Object 'apiVersion' is missing in 'kind: CustomResourceDefinition

    Object 'apiVersion' is missing in 'kind: CustomResourceDefinition

    What happened?

    Trying to install provider after building and pushing to dockerhub

    kubectl crossplane install provider ajwportalteam/provider-jet-azuread-amd64:v0.1.0
    

    But providerrevision object reports

    image

    Did I miss a step in the guide? make run is fine

    How can we reproduce it?

    Use same image / source at https://github.com/ajwgroup/provider-jet-azuread

    bug 
    opened by mikebollandajw 15
  • Need tip with converting ldap provider

    Need tip with converting ldap provider

    Im trying to convert ldap provider(https://registry.terraform.io/providers/Ouest-France/ldap/latest/docs) there: https://github.com/wNvSp1r1T/terraform-jet-ldap/commit/8dfc0a9c1c41559ce91e0bacf25d4340c9b7b85c

    And i'm always get error: "summary":"LDAP Result Code 34 \"Invalid DN Syntax\" 1.664437610235876e+09 DEBUG provider-jet-ldap Cannot observe external resource {"controller": "managed/ldapgroup.ldap.jet.crossplane.io/v1alpha1, kind=group", "request": "/hello-crossplane", "uid": "574384b6-1ec8-4bd9-9f1c-4b08821b7adb", "version": "208755", "external-name": "hello-crossplane", "error": "cannot run refresh: readObjectStart: expect { or n, but found 2, error found in #1 byte of ...

    my examples/providerconfig/providerconfig.yaml is default one my examples/providerconfig/secret.yaml is

        {
          "host": "ldap.ru",
          "port": "389",
          "bind_user": "bind_user",
          "bind_password": "test-ldap"
        }
    

    my examples/ldapgroup/ldapgroup.yaml is:

    kind: Group
    metadata:
      name: hello-crossplane
    spec:
      forProvider:
        description: "Managed with Crossplane ldap Provider (generated with Terrajet)"
        ou: "OU=test,OU=sre,OU=Resources,OU=Main,DC=X,DC=eu
      providerConfigRef:
        name: default
    

    I will be grateful for your help

    bug 
    opened by IvanTurgenev1 12
  • Load test with 100+ CR instances

    Load test with 100+ CR instances

    What problem are you facing?

    Common controller works for our prototypes but we need to do more experiments to see its scalability. There are certain issues that need some data before taking action, like https://github.com/crossplane-contrib/terrajet/issues/38

    How could Terrajet help solve your problem?

    We can have a complex composition with 10+ resources and create 10 XRs using that. Then check the resource usage and errors across the board to see if we hit any limit. We can define some limits in the deployment of the controller and see at what point we start to see context deadline exceeded errors, meaning we can't complete a single reconciliation pass with those limits.

    enhancement alpha 
    opened by muvaf 9
  • Consider making Terraform CLI talk to the single provider server

    Consider making Terraform CLI talk to the single provider server

    What problem are you facing?

    Right now we have a single provider binary and all terraform invocations use that one but each of them spins up a new provider server to talk to. We haven't observed noticeable issues but it's likely that this will cause performance issues.

    How could Terrajet help solve your problem?

    @ulucinar discovered a way to make Terraform CLI talk to the existing provider server. He didn't feel great about it for the initial pass but we might need to reconsider it after testing the providers with 100+ custom resources using composition.

    enhancement beta 
    opened by muvaf 7
  • Support for configuring blocks with optional parameters in attributes-as-blocks mode

    Support for configuring blocks with optional parameters in attributes-as-blocks mode

    Description of your changes

    Fixes #232

    With this PR, configuring blocks with optional parameters will be possible. In terraform schemas, this logic works with a field: ConfigMode. If the value of the ConfigMode field is SchemaConfigModeAttr, we have to provide values for all options. Please see the details: https://stackoverflow.com/questions/69079945/terraform-inappropriate-value-for-attribute-ingress-while-creating-sg/69080432#69080432

    Reference document: https://www.terraform.io/language/attr-as-blocks

    In this PR, a new check was added for tfTag. If the parent field is in the SchemaConfigModeAttr, the omitempty tag is deleted from tftags of subfields.

    I have:

    • [x] Read and followed Crossplane's contribution process.
    • [x] Run make reviewable to ensure this PR is ready for review.
    • [x] Added backport release-x.y labels to auto-backport this PR if necessary.

    How has this code been tested

    This PR was tested in provider-jet-aws side by provisioning, updating and deleting SecurityGroup resource. In tests the manifest was used:

    apiVersion: ec2.aws.jet.crossplane.io/v1alpha2
    kind: SecurityGroup
    metadata:
        name: example-securitygroup
    spec:
      forProvider:
        name: example-securitygroup-sg
        description: "i am a description"
        region: us-west-2
        vpcId: vpc-somevpc
        ingress:
        - fromPort: 6379
          toPort: 6379
          protocol: tcp
          description: "i am also a description"
          cidrBlocks:
          - 10.0.0.0/16
          ipv6CidrBlocks: []
          self: false
          prefixListIds: []
          securityGroups: []
        tags:
          managed-by: crossplane
      providerConfigRef:
        name: jet-aws-provider
    

    Please see the original issue in jet-aws side: https://github.com/crossplane-contrib/provider-jet-aws/issues/157

    backport release-0.4 
    opened by sergenyalcin 6
  • Scale Test Shared gRPC server-based Implementations

    Scale Test Shared gRPC server-based Implementations

    What problem are you facing?

    We have produced a shared gRPC server-based implementation for provider-jet-azure in the context of https://github.com/crossplane/terrajet/issues/38. The provider-jet-azure packages ulucinar/provider-jet-azure-arm64:shared-grpc and ulucinar/provider-jet-azure-amd64:shared-grpc are modified to run the terraform-provider-azurerm binary plugin in the background as a shared gRPC server and the Terraform CLI does not have to fork the binary plugin for each of its requests.

    How could Terrajet help solve your problem?

    Similar to what we have done previously in https://github.com/crossplane/terrajet/issues/55, we need to reevaluate the performance of [email protected] and also the shared gRPC implementation using the above provider packages. This will allow us to assess and quantify any performance improvements with the shared gRPC implementation. Some of the previously used scripts for #55 are available in https://github.com/ulucinar/terrajet-scale.

    enhancement 
    opened by ulucinar 6
  • Conversion from tfjson to consume schema from cli output

    Conversion from tfjson to consume schema from cli output

    Description of your changes

    Fixes #175 in the scope of #131

    I have:

    • [x] Read and followed Crossplane's contribution process.
    • [x] Run make reviewable to ensure this PR is ready for review.
    • [ ] Added backport release-x.y labels to auto-backport this PR if necessary.

    How has this code been tested

    See the provider-gcp schema generated using the conversion here: https://github.com/crossplane-contrib/provider-jet-gcp/pull/38/

    Tested with provider-github with the updated jet template in this PR: https://github.com/crossplane-contrib/provider-jet-template/pull/13

    opened by turkenh 6
  • Pipeline abstraction and configuration interface improvements

    Pipeline abstraction and configuration interface improvements

    Description of your changes

    This PR abstracts the terrajet code running generation pipelines from tf providers. With the changes here, terrajet based providers would only need to provide a Provider configuration (with resource-specific configurations) and Terrajet will take care of the rest.

    This PR also makes some (breaking) changes to interact with Terrajet aiming for a more flexible (and hopefully more stable) interface.

    Finally adds support for generating Terraform Providers depending on an old version (<v2) of Terraform plugin SDK.

    Fixes #61 Fixes #129

    I have:

    • [x] Read and followed Crossplane's contribution process.
    • [x] Run make reviewable to ensure this PR is ready for review.

    How has this code been tested

    Tested with autoscaling and ec2 groups in provider-tf-aws using this PR:

    1. Run make generate
    2. Verify that there is no unexpected changes in the generated code.
    opened by turkenh 6
  • Memory-based bookkeeping for Terraform operations

    Memory-based bookkeeping for Terraform operations

    Description of your changes

    Changes the bookkeeping of the operations from xp.lock and .store files to a global map of operations WorkspaceStore. In addition, there are a couple of things in this PR that's not directly related to bookeeping but help the flow. Main changes are as following:

    • New WorkspaceStore to hold all Workspace objects that is created per CR instance, identified by their UUIDs.
    • Handles Terraform crashes by removing Terraform state lock file if the last operation finished (crash, error or success).
    • It spawns a new go routine for each async execution and runs the given callback once finished.
    • The callback of Apply operation is to update the annotations to make sure we capture the server-side computed IDs as soon as possible and get enqueued.
    • Introduces new optional synchronous versions of Apply and Destroy operations for resources whose creation take short time, letting us save the state as fast as it can be.
    • Assumes that if terraform destroy completes successfully, the resource is gone so that we don't have to implement custom logic to handle resources whose read calls find the resource even after it's actually deleted (KMS keys, some configuration CRs)
    • Makes the TF client such that you don't have to call ApplyAsync again to poll its results, allowing us to use the latest crossplane-runtime.
    • [TBD] Moves the adapter into the ExternalClient implementation directly. There is only a single interface now, similar to native providers.
    • Reorganizes the folder structure to be similar to native providers, i.e. terraform is client package and controller is the managed reconciler implementation package.
    • Since the tracking is in-memory start/stop of the controller does not end up in a locked state.

    This PR is mostly accumulation of our collective experience testing the controllers during implementation of features during the initial push that we released as tech preview.

    Special shoutout to @ulucinar who had to deal with this complex beast in the initial rush. The state handling logic he implemented is still the core logic with only some refactoring that incorporated our understanding of the whole thing.

    It's ready for review but there are TODOs before merging:

    • Replace osfile calls with afero for easier testing.
    • Unit tests for controller.
    • Unit tests for terraform package.

    Fixes https://github.com/crossplane-contrib/terrajet/issues/74 Fixes https://github.com/crossplane-contrib/terrajet/issues/39 Fixes https://github.com/crossplane-contrib/terrajet/issues/22 Fixes https://github.com/crossplane-contrib/terrajet/issues/21

    I have:

    • [x] Read and followed Crossplane's contribution process.
    • [x] Run make reviewable to ensure this PR is ready for review.
    • [x] Added backport release-x.y labels to auto-backport this PR if necessary.

    How has this code been tested

    I have tested it in both async and sync mode using provider-tf-aws. The example YAMLs will be there. Additionally, I tested with stopping/starting the controller and it was able to continue without any errors.

    Sync:

    apiVersion: vpc.aws.tf.crossplane.io/v1alpha1
    kind: Vpc
    metadata:
      name: sample-vpc
    spec:
      forProvider:
        region: us-west-1
        cidrBlock: 10.0.0.0/16
        tagsAll:
          Name: muvaftest
        tags:
          Name: muvaftest
      providerConfigRef:
        name: example
    

    It works just like other providers. There are a few gotchas. For example, after the deletion is completed we requeue immediately but for some reason that version of custom resource is old, hence, the finalizer removal Update fails but at that point we removed the directory. So, in the next reconcile we init again, check existence and then finalizer is removed successfully. Though this looks like a general problem that should be addressed in controller-runtime or crossplane-runtime. Other than that, I haven't caught an obvious problem.

    Async:

    apiVersion: rds.aws.tf.crossplane.io/v1alpha1
    kind: RdsCluster
    metadata:
      name: sample-cluster
    spec:
      forProvider:
        region: us-west-1
        engine: aurora-postgresql
        masterUsername: cpadmin
        masterPassword: testPass23!
        skipFinalSnapshot: true
      providerConfigRef:
        name: example
    

    This one also works as expected. One thing different regarding the async mode is that we introdue a new condition to report the last operation status. This would come in the schema of the resource for other providers but since we control the client, we return it as part of RefreshResult and put it under a new condition separately.

    One thing I don't love about async mode is that even if the operation takes 5 seconds due to an error, we have to wait for a full poll interval in order to be aware of the error and I couldn't find a way to manually enqueue the resource. Same thing happens with native provider async resources as well but they usually do a client side validation with their async Create call.

    I have a hunch that we might be able to get some simplification if we separate sync/async into two ExternalClient implementations but I held that off for now.

    opened by muvaf 6
  • Add late-initialization for Terraformed resources

    Add late-initialization for Terraformed resources

    Description of your changes

    Introduces late-initialization for Terraformed resources. Copies late-initialization library from provider-aws pkg/controller/cloudfront/ package.

    Fixes #10

    I have:

    • [X] Read and followed Crossplane's contribution process.
    • [X] Run make reviewable to ensure this PR is ready for review.
    • [ ] Added backport release-x.y labels to auto-backport this PR if necessary.

    How has this code been tested

    I have manually tested this PR with provider-tf-azure. When a VirtualNetwork MR with the following manifest:

    apiVersion: virtual.azure.tf.crossplane.io/v1alpha1
    kind: VirtualNetwork
    metadata:
      name: example
    spec:
      deletionPolicy: Orphan
      forProvider:
        addressSpace:
        - 10.0.0.0/16
        dnsServers:
        - 10.0.0.1
        - 10.0.0.2
        - 10.0.0.3
        location: East US
        name: importDemo
        resourceGroupName: alper
        tags:
          purpose: demo
          key1: value1
      providerConfigRef:
        name: example
    

    is provisioned and then deleted, and after a new VirtualNetwork MR is imported using the following manifest:

    apiVersion: virtual.azure.tf.crossplane.io/v1alpha1
    kind: VirtualNetwork
    metadata:
      name: imported
      annotations:
        crossplane.io/external-name: /subscriptions/.../resourceGroups/alper/providers/Microsoft.Network/virtualNetworks/importDemo
    spec:
      forProvider:
        name: "importDemo"
        location: "East US"
        resourceGroupName: "alper"
        addressSpace:
          - "10.0.0.0/16"
      providerConfigRef:
        name: example
    

    , the imported MR is late-initialized as follows:

    apiVersion: virtual.azure.tf.crossplane.io/v1alpha1
    kind: VirtualNetwork
    metadata:
      annotations:
        crossplane.io/external-name: /subscriptions/038f2b7c-3265-43b8-8624-c9ad5da610a8/resourceGroups/alper/providers/Microsoft.Network/virtualNetworks/importDemo
        creationTimestamp: "2021-09-10T03:18:29Z"
    ...
      finalizers:
      - finalizer.managedresource.crossplane.io
      name: imported
    ...
    spec:
      deletionPolicy: Delete
      forProvider:
        addressSpace:
        - 10.0.0.0/16
        dnsServers:
        - 10.0.0.1
        - 10.0.0.2
        - 10.0.0.3
        location: East US
        name: importDemo
        resourceGroupName: alper
        tags:
          key1: value1
          purpose: demo
      providerConfigRef:
        name: example
    status:
      atProvider:
        guid: 1222d5b8-41b7-4d84-bca0-a8282390422e
      conditions:
      - lastTransitionTime: "2021-09-10T03:18:39Z"
        reason: Available
        status: "True"
        type: Ready
      - lastTransitionTime: "2021-09-10T03:18:39Z"
        reason: ReconcileSuccess
        status: "True"
        type: Synced
    
    opened by ulucinar 6
  • Support configuring the json tags of a field

    Support configuring the json tags of a field

    What problem are you facing?

    Currently, when an MR is generated from Terrajet, the JSON tags are a consequence of the field type. If it is required, it doesn't have omitempty. If it's optional, it has the omitempty tag. Here is the code that decides it:

    https://github.com/crossplane/terrajet/blob/8d0ed485f9511b65a8f3a83801092bcae60678dd/pkg/types/builder.go#L273-L279

    This is fine for most use cases for managed resources. But there are some APIs and fields that would be benefited if this behavior can be configured.


    The practical use case that I have for this is with the BranchProtection managed resource from GitHub terraform provider.

    apiVersion: branchprotection.github.jet.crossplane.io/v1alpha1
    kind: BranchProtection
    metadata:
      name: main-branch
    spec:
      deletionPolicy: Delete
      forProvider:
        repositoryId: my-repository
        pattern: main
        requiredStatusChecks:
          - strict: false
            contexts:
              - my-pipeline-check
      providerConfigRef:
        name: github-jet-provider-config
    

    In the YAML above, the spec.forProvider.requiredStatusChecks[0].contexts field is optional (so the JSON tag has the omitempty value). It configures which status checks are mandatory to pass before a PR merge can be done.

    image

    By applying the YAML above, we expect that the status check my-pipeline-check MUST pass before any PR could be merged. This works as expected.

    The problem is that, if I change the contexts field to contexts: [] -- for example:

    apiVersion: branchprotection.github.jet.crossplane.io/v1alpha1
    kind: BranchProtection
    metadata:
      name: main-branch
    spec:
      deletionPolicy: Delete
      forProvider:
        repositoryId: my-repository
        pattern: main
        requiredStatusChecks:
          - strict: false
            contexts: []
      providerConfigRef:
        name: github-jet-provider-config
    

    I would expect that the external API would also have an empty list of status checks. But the Managed Resource code doesn't remove the my-pipeline-check status check from the API.

    How could Terrajet help solve your problem?

    My hypothesis is that the code is considering the contexts field as not_specified, so it isn't actively forcing the BranchProtection (in the external API) to not have the contexts field filled.

    If the hypothesis is correct, we could configure this tags and the code could differ from not_specified, specified and empty_value and configures the external API accordingly.

    enhancement 
    opened by Feggah 0
  • Cannot generate provider crds

    Cannot generate provider crds

    What happened?

    I'm trying to generate CRDS with make generate with custom provider

    cannot generate crd for resource lb_instance: cannot build types for Instance: cannot build the Types: cannot infer type from schema of field listeners: invalid schema type TypeInvalid
    

    there is my json schema:

                  "listeners": {
                    "nested_type": {
                      "attributes": {
                        "connection_limit": {
                          "type": "number",
                          "description_kind": "plain",
                          "required": true
                        },
                        "description": {
                          "type": "string",
                          "description_kind": "plain",
                          "computed": true
                        },
                        "headers": {
                          "nested_type": {
                            "attributes": {
                              "x_forwarded_for": {
                                "type": "bool",
                                "description_kind": "plain",
                                "optional": true
                              },
                              "x_forwarded_port": {
                                "type": "bool",
                                "description_kind": "plain",
                                "optional": true
                              },
                              "x_forwarded_proto": {
                                "type": "bool",
                                "description_kind": "plain",
                                "optional": true
                              }
                            },
                            "nesting_mode": "single"
                          },
                          "description_kind": "plain",
                          "optional": true
                        },
                        "id": {
                          "type": "string",
                          "description_kind": "plain",
                          "computed": true
                        },
                        "name": {
                          "type": "string",
                          "description": "displayed name",
                          "description_kind": "plain",
                          "required": true
                        },
                        "pool": {
                          "nested_type": {
                            "attributes": {
                              "algorithm": {
                                "type": "string",
                                "description": "\"LEAST_CONNECTIONS\", \"ROUND_ROBIN\", \"SOURCE_IP\"",
                                "description_kind": "plain",
                                "required": true
                              },
                              "description": {
                                "type": "string",
                                "description_kind": "plain",
                                "computed": true
                              },
                              "id": {
                                "type": "string",
                                "description_kind": "plain",
                                "computed": true
                              },
                              "members": {
                                "nested_type": {
                                  "attributes": {
                                    "address": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "backup": {
                                      "type": "bool",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "monitor_address": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "optional": true
                                    },
                                    "monitor_port": {
                                      "type": "number",
                                      "description_kind": "plain",
                                      "optional": true
                                    },
                                    "name": {
                                      "type": "string",
                                      "description": "name of the existed instance",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "network_name": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "port": {
                                      "type": "number",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "weight": {
                                      "type": "number",
                                      "description_kind": "plain",
                                      "required": true
                                    }
                                  },
                                  "nesting_mode": "list"
                                },
                                "description_kind": "plain",
                                "required": true
                              },
                              "monitor": {
                                "nested_type": {
                                  "attributes": {
                                    "delay": {
                                      "type": "number",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "expected_codes": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "optional": true
                                    },
                                    "id": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "computed": true
                                    },
                                    "max_retries": {
                                      "type": "number",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "max_retries_down": {
                                      "type": "number",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "name": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "optional": true
                                    },
                                    "timeout": {
                                      "type": "number",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "type": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "required": true
                                    },
                                    "url_path": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "optional": true
                                    }
                                  },
                                  "nesting_mode": "single"
                                },
                                "description_kind": "plain",
                                "optional": true
                              },
                              "name": {
                                "type": "string",
                                "description_kind": "plain",
                                "computed": true
                              },
                              "session_persistence": {
                                "nested_type": {
                                  "attributes": {
                                    "cookie_name": {
                                      "type": "string",
                                      "description_kind": "plain",
                                      "optional": true
                                    },
                                    "type": {
                                      "type": "string",
                                      "description": "\"NONE\", \"HTTP_COOKIE\", \"APP_COOKIE\", \"SOURCE_IP\"",
                                      "description_kind": "plain",
                                      "required": true
                                    }
                                  },
                                  "nesting_mode": "single"
                                },
                                "description_kind": "plain",
                                "required": true
                              }
                            },
                            "nesting_mode": "single"
                          },
                          "description_kind": "plain",
                          "required": true
                        },
                        "port": {
                          "type": "number",
                          "description_kind": "plain",
                          "required": true
                        },
                        "protocol": {
                          "type": "string",
                          "description": "\"HTTP\", \"HTTPS\", \"TCP\", \"UDP\", \"TERMINATED_HTTPS\"",
                          "description_kind": "plain",
                          "required": true
                        },
                        "timeout_client_data": {
                          "type": "number",
                          "description_kind": "plain",
                          "optional": true
                        },
                        "timeout_member_connect": {
                          "type": "number",
                          "description_kind": "plain",
                          "optional": true
                        },
                        "timeout_member_data": {
                          "type": "number",
                          "description_kind": "plain",
                          "optional": true
                        },
                        "timeout_tcp_inspect": {
                          "type": "number",
                          "description_kind": "plain",
                          "optional": true
                        },
                        "tls_container_id": {
                          "type": "string",
                          "description_kind": "plain",
                          "optional": true
                        }
                      },
                      "nesting_mode": "list"
                    },
                    "description_kind": "plain",
                    "required": true
                  }
    

    Can you please give me a tip what wrong with this schema and how i can fix this issue?

    bug 
    opened by IvanTurgenev1 2
  • Using

    Using "Test":true in TF_REATTACH_PROVIDER cause memory leaks

    What happened?

    Hi, This is more like JFYI. When client connects to provider with "Test":true in TF_REATTACH_PROVIDERS, it will cause a memory leak. More info here: https://github.com/hashicorp/terraform-provider-aws/issues/26130 https://github.com/hashicorp/go-plugin/issues/215

    How can we reproduce it?

    The easy way to reproduce is to run terraform-provider-aws --debug, and constantly run terraform plan against this provider in debug mode.

    bug 
    opened by RealFatCat 0
  • Allow prevent_destroy for provider

    Allow prevent_destroy for provider

    I'm trying to use https://registry.terraform.io/providers/pan-net/powerdns/latest as crossplane terrajet provider and when i'm trying to change A records by adding multiple values for A records: from

    apiVersion: record.powerdns.jet.crossplane.io/v1alpha1
    kind: Record
    metadata:
      name: example
    spec:
      forProvider:
       zone: "example.com."
       name: "www.example.com."
       type: "A"
       ttl: 300
       records: ["192.168.0.11"]
      providerConfigRef:
        name: powerdns
    

    to

    apiVersion: record.powerdns.jet.crossplane.io/v1alpha1
    kind: Record
    metadata:
      name: example
    spec:
      forProvider:
       zone: "example.com."
       name: "www.example.com."
       type: "A"
       ttl: 300
       records: ["192.168.0.11", "192.168.0.12"]
      providerConfigRef:
        name: powerdns
    

    error happens: "Resource powerdns_record.example has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed."

    How can i resolve this problem? is it possibly to set prevent_destroy=fasle somehow?

    bug 
    opened by IvanTurgenev1 0
  • Import cycle not allowed

    Import cycle not allowed

    What happened?

    I am having some "import cycle" issues when running make reviewable on my Terrajet project. Below is three resources that generates the import cycle not allowed error.

    • Team Routing Rule resource references.

      • team_id => type Team
      • notify.id => type Escalation
    • Escalation resource references,

      • owner_team_id => type Team
      • rules.recipient.id => type Team

    image

    This will generate the following error on running make reviewable.

    WARN [runner] Can't run linter goanalysis_metalinter: bodyclose: failed prerequisites: [[email protected]/ok-amba/provider-jet-opsgenie/apis/team/v1alpha1: analysis skipped: errors in package: [/home/kerwood/Git/provider-jet-opsgenie/apis/team/v1alpha1/zz_generated.resolvers.go:23:11: could not import github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1 (-: import cycle not allowed: import stack: [github.com/ok-amba/provider-jet-opsgenie/apis github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1 github.com/ok-amba/provider-jet-opsgenie/apis/team/v1alpha1 github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1])]] 
    WARN [runner] Can't run linter unused: buildir: analysis skipped: errors in package: [-: import cycle not allowed: import stack: [github.com/ok-amba/provider-jet-opsgenie/apis github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1 github.com/ok-amba/provider-jet-opsgenie/apis/team/v1alpha1 github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1]] 
    ERRO Running error: buildir: analysis skipped: errors in package: [-: import cycle not allowed: import stack: [github.com/ok-amba/provider-jet-opsgenie/apis github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1 github.com/ok-amba/provider-jet-opsgenie/apis/team/v1alpha1 github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1]] 
    11:35:52 [FAIL]
    make[2]: *** [build/makelib/golang.mk:194: go.lint] Error 1
    make[1]: *** [build/makelib/common.mk:357: lint] Error 2
    make: *** [build/makelib/common.mk:425: reviewable] Error 2
    

    My guess is that its importing github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1 multiple times for some reason.

    [
     github.com/ok-amba/provider-jet-opsgenie/apis 
     github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1 
     github.com/ok-amba/provider-jet-opsgenie/apis/team/v1alpha1 
     github.com/ok-amba/provider-jet-opsgenie/apis/escalation/v1alpha1
    ]
    

    Links to files

    Team Routing Rule config.go file: https://github.com/ok-amba/provider-jet-opsgenie/blob/1d82bf7f2d5477d61db916c4868333187a0b10b1/config/teamroutingrule/config.go Escalation config.go file: https://github.com/ok-amba/provider-jet-opsgenie/blob/1d82bf7f2d5477d61db916c4868333187a0b10b1/config/escalation/config.go

    How can we reproduce it?

    git clone [email protected]:ok-amba/provider-jet-opsgenie.git
    cd provider-jet-opsgenie
    git checkout 1d82bf7f2d5477d61db916c4868333187a0b10b1
    make submodules
    make reviewable
    
    bug 
    opened by Kerwood 1
  • Terrajet providers take a long time on cluster with many resources

    Terrajet providers take a long time on cluster with many resources

    Hello team!

    Me and my team are developing a solution for developers to be able to manage infrastructure resources through an API. This solution relies on crossplane to manage cloud resources as kubernetes CRs.

    We've started using terrajet for AWS, but changed to the native provider because of this issue. New we also need to manage other resources for which there are no native providers (namely vault, boundary and datadog) and it would be great to use terrajet.

    Existing Context

    initial slack thread with problem description

    What happened?

    The expected behaviour is that terrajet resources time-to-readiness wouldn't depend on the amount of resources that already exist in the cluster.

    In reality, the terrajet controller takes roughly 3 * (number of existing resources) * (time it takes for one resource to be reconciled) seconds to consider a new resource ready.

    The examples in this issue are using provider-jet-aws, but all terrajet providers have the same behaviour, from what I can see.

    Below are the debug logs related to one recently created policy (terrajetloadtestslow). It was created on a provider that is already managing 200 other aws policies.

    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630252994777613e+09       DEBUG   provider-jet-aws        Reconciling     {"controller": "managed/iam.aws.jet.crossplane.io/v1alpha2, kind=policy", "request": "/terrajetloadtestslow"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630253028353422e+09       DEBUG   provider-jet-aws        refresh ended   {"workspace": "/tmp/636bbd3e-94bb-40ae-acde-f866e095b0e6", "out": "{\"@level\":\"info\",\"@message\":\"Terraform 1.0.5\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:20.558331Z\",\"terraform\":\"1.0.5\",\"type\":\"version\",\"ui\":\"0.1.0\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Refreshing state...\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:22.822179Z\",\"hook\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"id_key\":\"id\"},\"type\":\"refresh_start\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Refresh complete\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:22.822992Z\",\"hook\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null}},\"type\":\"refresh_complete\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Drift detected (delete)\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:22.823374Z\",\"change\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"action\":\"delete\"},\"type\":\"resource_drift\"}\n{\"@level\":\"info\",\"@message\":\"Plan: 0 to add, 0 to change, 0 to destroy.\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:22.823487Z\",\"changes\":{\"add\":0,\"change\":0,\"remove\":0,\"operation\":\"plan\"},\"type\":\"change_summary\"}\n{\"@level\":\"info\",\"@message\":\"Apply complete! Resources: 0 added, 0 changed, 0 destroyed.\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:22.829034Z\",\"changes\":{\"add\":0,\"change\":0,\"remove\":0,\"operation\":\"apply\"},\"type\":\"change_summary\"}\n{\"@level\":\"info\",\"@message\":\"Outputs: 0\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:22.829081Z\",\"outputs\":{},\"type\":\"outputs\"}\n"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630253059385006e+09       DEBUG   provider-jet-aws        apply ended     {"workspace": "/tmp/636bbd3e-94bb-40ae-acde-f866e095b0e6", "out": "{\"@level\":\"info\",\"@message\":\"Terraform 1.0.5\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:22.897931Z\",\"terraform\":\"1.0.5\",\"type\":\"version\",\"ui\":\"0.1.0\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Plan to create\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:25.127798Z\",\"change\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"action\":\"create\"},\"type\":\"planned_change\"}\n{\"@level\":\"info\",\"@message\":\"Plan: 1 to add, 0 to change, 0 to destroy.\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:25.127975Z\",\"changes\":{\"add\":1,\"change\":0,\"remove\":0,\"operation\":\"plan\"},\"type\":\"change_summary\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Creating...\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:25.630013Z\",\"hook\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"action\":\"create\"},\"type\":\"apply_start\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Creation complete after 0s [id=arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow]\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:25.924980Z\",\"hook\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"action\":\"create\",\"id_key\":\"id\",\"id_value\":\"arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow\",\"elapsed_seconds\":0},\"type\":\"apply_complete\"}\n{\"@level\":\"info\",\"@message\":\"Apply complete! Resources: 1 added, 0 changed, 0 destroyed.\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:25.932091Z\",\"changes\":{\"add\":1,\"change\":0,\"remove\":0,\"operation\":\"apply\"},\"type\":\"change_summary\"}\n{\"@level\":\"info\",\"@message\":\"Outputs: 0\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:28:25.932195Z\",\"outputs\":{},\"type\":\"outputs\"}\n"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.663025305947192e+09        DEBUG   provider-jet-aws        Successfully requested creation of external resource {"controller": "managed/iam.aws.jet.crossplane.io/v1alpha2, kind=policy", "request": "/terrajetloadtestslow", "uid": "636bbd3e-94bb-40ae-acde-f866e095b0e6", "version": "573953565", "external-name": "", "external-name": "arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630253059475644e+09       DEBUG   events  Normal  {"object": {"kind":"Policy","name":"terrajetloadtestslow","uid":"636bbd3e-94bb-40ae-acde-f866e095b0e6","apiVersion":"iam.aws.jet.crossplane.io/v1alpha2","resourceVersion":"573965831"}, "reason": "CreatedExternalResource", "message": "Successfully requested creation of external resource"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.663025826864625e+09        DEBUG   provider-jet-aws        Reconciling     {"controller": "managed/iam.aws.jet.crossplane.io/v1alpha2, kind=policy", "request": "/terrajetloadtestslow"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630258282078466e+09       DEBUG   provider-jet-aws        refresh ended   {"workspace": "/tmp/636bbd3e-94bb-40ae-acde-f866e095b0e6", "out": "{\"@level\":\"info\",\"@message\":\"Terraform 1.0.5\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:37:06.895864Z\",\"terraform\":\"1.0.5\",\"type\":\"version\",\"ui\":\"0.1.0\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Refreshing state... [id=arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow]\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:37:08.112383Z\",\"hook\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"id_key\":\"id\",\"id_value\":\"arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow\"},\"type\":\"refresh_start\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Refresh complete [id=arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow]\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:37:08.197666Z\",\"hook\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"id_key\":\"id\",\"id_value\":\"arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow\"},\"type\":\"refresh_complete\"}\n{\"@level\":\"info\",\"@message\":\"Plan: 0 to add, 0 to change, 0 to destroy.\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:37:08.198356Z\",\"changes\":{\"add\":0,\"change\":0,\"remove\":0,\"operation\":\"plan\"},\"type\":\"change_summary\"}\n{\"@level\":\"info\",\"@message\":\"Apply complete! Resources: 0 added, 0 changed, 0 destroyed.\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:37:08.203956Z\",\"changes\":{\"add\":0,\"change\":0,\"remove\":0,\"operation\":\"apply\"},\"type\":\"change_summary\"}\n{\"@level\":\"info\",\"@message\":\"Outputs: 0\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:37:08.203989Z\",\"outputs\":{},\"type\":\"outputs\"}\n"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630258282080562e+09       DEBUG   provider-jet-aws        External resource is up to date {"controller": "managed/iam.aws.jet.crossplane.io/v1alpha2, kind=policy", "request": "/terrajetloadtestslow", "uid": "636bbd3e-94bb-40ae-acde-f866e095b0e6", "version": "573965832", "external-name": "arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow", "requeue-after": 1663025888.2080543}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630263118990378e+09       DEBUG   provider-jet-aws        Reconciling     {"controller": "managed/iam.aws.jet.crossplane.io/v1alpha2, kind=policy", "request": "/terrajetloadtestslow"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630263132372632e+09       DEBUG   provider-jet-aws        refresh ended   {"workspace": "/tmp/636bbd3e-94bb-40ae-acde-f866e095b0e6", "out": "{\"@level\":\"info\",\"@message\":\"Terraform 1.0.5\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:45:11.952075Z\",\"terraform\":\"1.0.5\",\"type\":\"version\",\"ui\":\"0.1.0\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Refreshing state... [id=arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow]\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:45:13.135934Z\",\"hook\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"id_key\":\"id\",\"id_value\":\"arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow\"},\"type\":\"refresh_start\"}\n{\"@level\":\"info\",\"@message\":\"aws_iam_policy.terrajetloadtestslow: Refresh complete [id=arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow]\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:45:13.226661Z\",\"hook\":{\"resource\":{\"addr\":\"aws_iam_policy.terrajetloadtestslow\",\"module\":\"\",\"resource\":\"aws_iam_policy.terrajetloadtestslow\",\"implied_provider\":\"aws\",\"resource_type\":\"aws_iam_policy\",\"resource_name\":\"terrajetloadtestslow\",\"resource_key\":null},\"id_key\":\"id\",\"id_value\":\"arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow\"},\"type\":\"refresh_complete\"}\n{\"@level\":\"info\",\"@message\":\"Plan: 0 to add, 0 to change, 0 to destroy.\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:45:13.227599Z\",\"changes\":{\"add\":0,\"change\":0,\"remove\":0,\"operation\":\"plan\"},\"type\":\"change_summary\"}\n{\"@level\":\"info\",\"@message\":\"Apply complete! Resources: 0 added, 0 changed, 0 destroyed.\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:45:13.232695Z\",\"changes\":{\"add\":0,\"change\":0,\"remove\":0,\"operation\":\"apply\"},\"type\":\"change_summary\"}\n{\"@level\":\"info\",\"@message\":\"Outputs: 0\",\"@module\":\"terraform.ui\",\"@timestamp\":\"2022-09-12T23:45:13.232734Z\",\"outputs\":{},\"type\":\"outputs\"}\n"}
    crossplane-provider-jet-aws-599c4ea494ca-55b9dd885f-zr2pb provider-jet-aws 1.6630263132466455e+09       DEBUG   provider-jet-aws        External resource is up to date {"controller": "managed/iam.aws.jet.crossplane.io/v1alpha2, kind=policy", "request": "/terrajetloadtestslow", "uid": "636bbd3e-94bb-40ae-acde-f866e095b0e6", "version": "573976982", "external-name": "arn:aws:iam::ACCOUNT_NUMBER:policy/terrajetloadtestslow", "requeue-after": 1663026373.2466433}
    

    From these logs we can also see the time it took for the resource to be Ready. From 23:28:19 to 23:45:13, ~17 minutes. (using GMT time to make it easier)

    It should also be noted that the resource is created in the cloud provider at 23:28:25 (12 seconds), when the logs say Successfully requested creation of external resource, but it takes a long time until the resource is marked as Ready in kubernetes

    The reason why it takes so long seems to be that resource change events are not given any priority over the normal reconciliation loop. The controller stays in an endless loop reconciling every resource it knows about, which is necessary to override external changes in the providers, but any resource that is added or changed in kubernetes doesn't get immediately picked up. Rather it seems to be added to the queue and only seen after all existing reconcile loops are done, then the resource is picked up, and a creation flow starts, but the result will only be seen on the next reconciliation loop. Once that comes, terrajet will pick up the resource's status from the terraform state and requeue once more, only after this last requeue the status will be updated to Ready.

    It's also worth noting that changing the MaxConcurrentReconciles configuration to something bigger, like 5 or 10 reduces the time by that factor, but the underlying behavior persists and all resources are reconciled before considering new resource or updates Ready.

    What I'm having trouble understanding is how this terrajet workflow differs from the native provider-aws one, as both seem to use the same Reconcile function from crossplane-runtime.

    Also, the UseAsync configuration for a resource doesn't seem to change anything. I've tried both for policies.iam and for repositories.ecr with the same behavior.


    How can we reproduce it?

    • Create a new kubernetes cluster (with kind or in the cloud).
    • Install crossplane
    • Install a terrajet provider (I'll use AWS because it can be easily compared with the native provider-aws)
    • Create a handful of resources to be managed by terrajet (I'll use ECR because it is quickly created and dont incur costs)
    • Wait until all resources are created and ready it will take some minutes, but a burst of resources is expected to take a bit. Althought it does take much longer than provider-aws for the same resource.
    • Create one more resource to be managed by terrajet

    The last step will take a long time, which is the problem this bug report is about.

    Open collapsible for reproducible commands
    # Create kind cluster
    kind create cluster --name terrajet-load --image kindest/node:v1.23.10
    
    # Install crossplane with helm
    kubectl create namespace crossplane-system
    helm repo add crossplane-stable https://charts.crossplane.io/stable
    helm repo update
    helm upgrade --install crossplane --namespace crossplane-system crossplane-stable/crossplane
    
    # Install AWS terrajet provider
    kubectl apply -f - <<YAML
    apiVersion: pkg.crossplane.io/v1
    kind: Provider
    metadata:
      name: crossplane-provider-jet-aws
    spec:
      controllerConfigRef:
        name: config
      package: crossplane/provider-jet-aws:v0.5.0
    ---
    apiVersion: pkg.crossplane.io/v1alpha1
    kind: ControllerConfig
    metadata:
      name: config
    spec:
      args:
      - --debug
    YAML
    ## use your aws credentials in the secret, if you have a custom way to interact with AWS, change the credentials key of this secret.
    kubectl create secret generic -n crossplane-system --from-file=credentials=$HOME/.aws/credentials aws-credentials
    kubectl apply -f - <<YAML
    apiVersion: aws.jet.crossplane.io/v1alpha1
    kind: ProviderConfig
    metadata:
      finalizers:
      - in-use.crossplane.io
      name: default
    spec:
      credentials:
        secretRef:
          key: credentials
          name: aws-credentials
          namespace: crossplane-system
        source: Secret
    YAML
    
    # Create a handful of resources managed by terrajet. I chose policies because they are created near-instantly in the cloud and don't incur costs
    ## Note: seq on MacOS doesn't seem to support the -w flag, it can be removed safely below
    for n in $(seq -w 200); do
      echo "---"
      sed "s/NUMBER/$n/" <<YAML
    apiVersion: iam.aws.jet.crossplane.io/v1alpha2
    kind: Policy
    metadata:
      name: terrajetloadtestNUMBER
    spec:
      forProvider:
        name: terrajetloadtestNUMBER
        policy: |
          {
            "Version": "2012-10-17",
            "Statement": [
              {
                "Effect": "Deny",
                "Action": "*",
                "Resource": ["*"]
              }
            ]
          }
      providerConfigRef:
        name: default
    YAML
    done | kubectl apply -f -
    
    # Wait until all are ready, it took about 15 minutes for me
    kubectl get policies.iam.aws.jet.crossplane.io
    
    # Create one more resource
    kubectl apply -f - <<YAML
    apiVersion: iam.aws.jet.crossplane.io/v1alpha2
    kind: Policy
    metadata:
      name: terrajetloadtestslow
    spec:
      forProvider:
        name: terrajetloadtestslow
        policy: |
          {
            "Version": "2012-10-17",
            "Statement": [
              {
                "Effect": "Deny",
                "Action": "*",
                "Resource": ["*"]
              }
            ]
          }
      providerConfigRef:
        name: default
    YAML
    
    # It takes a long time for the resource to become ready.
    kubectl get policies.iam.aws.jet.crossplane.io terrajetloadtestslow
    
    ########
    
    # Cleanup cloud resources
    kubectl delete policy.iam.aws.jet $(kubectl get policy.iam.aws.jet | grep terrajetloadtest | awk '{print $1}')
    
    # Delete kind cluster
    kind destroy cluster --name terrajet-load
    

    Possibly related discussions


    Please let me know if I can help figure out what's happening.

    bug 
    opened by Kasama 2
Releases(v0.4.2)
  • v0.4.2(Mar 7, 2022)

    What's Changed

    • [Backport release-0.4] Support for configuring blocks with optional parameters in attributes-as-blocks mode by @github-actions in https://github.com/crossplane/terrajet/pull/254
    • [Backport release-0.4] Fix sensitive path mapping by @github-actions in https://github.com/crossplane/terrajet/pull/256

    Full Changelog: https://github.com/crossplane/terrajet/compare/v0.4.1...v0.4.2

    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Mar 4, 2022)

    What's Changed

    • [Backport release-0.4] Cli Schema Conversion Fixes & Set ConfigMode Attribute by @github-actions in https://github.com/crossplane/terrajet/pull/252

    Full Changelog: https://github.com/crossplane/terrajet/compare/v0.4.0...v0.4.1

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Feb 22, 2022)

    Notable Changes

    With https://github.com/crossplane/terrajet/pull/174 , you don't need to import TF provider Go module anymore! We'll be able to run Terraform CLI to get the schema JSON and work with that directly, hence no need to deal with all dependency problems with the TF provider!

    With https://github.com/crossplane/terrajet/pull/198 , you can add your own initializers and there is a built-in common one that you can use off the shelf to tag your resources automatically! Take a look at the API Patterns doc to see why this is important.

    Breaking Changes

    • https://github.com/crossplane/terrajet/pull/195 changes the function signature of generated Setup function. A PR that does all the steps described below is here.
      • You'll need to call controller.Setup in your cmd/provider/main.go with the new Options struct.
      • The Setup signature of non-generated controllers will need to be changed as well, such as ProviderConfig controller.
      • In the new kubebuilder version, crd:trivialVersions=true argument has been removed, so you'll need to remove it from the command in your apis/generate.go
      • This change will update kubebuilder as well and in its new version it doesn't print extra two lines at the top anymore. So, in your Makefile, delete the crds.clean target and the line generate.done: crds.clean.

    What's Changed

    • controller: set ongoing operation condition correctly by @muvaf in https://github.com/crossplane/terrajet/pull/194
    • Update Configuring a Resource guide by @turkenh in https://github.com/crossplane/terrajet/pull/187
    • Conversion: Default to string as element type by @turkenh in https://github.com/crossplane/terrajet/pull/199
    • Fix function name in documentation by @ezgidemirel in https://github.com/crossplane/terrajet/pull/202
    • Terrajet provider generation documentation fixes by @ytsarev in https://github.com/crossplane/terrajet/pull/205
    • Tagging Convention by @sergenyalcin in https://github.com/crossplane/terrajet/pull/198
    • Support []string and []*string as sensitive fields by @sergenyalcin in https://github.com/crossplane/terrajet/pull/197
    • Add Sergen as a maintainer by @turkenh in https://github.com/crossplane/terrajet/pull/215
    • Add unit tests for builder by @sergenyalcin in https://github.com/crossplane/terrajet/pull/189
    • Support for map[string]*string as sensitive fields by @sergenyalcin in https://github.com/crossplane/terrajet/pull/218
    • Fixes the unmarshal error for sensitive string type by @sergenyalcin in https://github.com/crossplane/terrajet/pull/238
    • Conversion from tfjson to consume schema from cli output by @turkenh in https://github.com/crossplane/terrajet/pull/174
    • Fix reconciliation errors if a referenced secret is deleted by @ezgidemirel in https://github.com/crossplane/terrajet/pull/237
    • Update go, crossplane-runtime and k8s libraries by @muvaf in https://github.com/crossplane/terrajet/pull/195
    • [Backport release-0.4] controller: set async finished condition only if the resource is async by @github-actions in https://github.com/crossplane/terrajet/pull/240

    New Contributors

    • @ezgidemirel made their first contribution in https://github.com/crossplane/terrajet/pull/202
    • @ytsarev made their first contribution in https://github.com/crossplane/terrajet/pull/205

    Full Changelog: https://github.com/crossplane/terrajet/compare/v0.3.0...v0.4.0

    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Jan 18, 2022)

    What's Changed

    No breaking changes, fixing an issue in conversion package which is used when terraform provider still uses an old version (i.e. v1) of terraform plugin sdk.

    • [cherry-pick to release-0.3] Conversion: Default to string as element type by @turkenh in https://github.com/crossplane/terrajet/pull/201

    Full Changelog: https://github.com/crossplane/terrajet/compare/v0.3.1...v0.3.2

    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Jan 7, 2022)

    Notable Changes

    The new async operation tracking condition wasn't working properly, https://github.com/crossplane/terrajet/pull/194 fixed it with no breaking changes.

    See https://github.com/crossplane/terrajet/releases/tag/v0.3.0 for full release notes of v0.3.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Jan 6, 2022)

    Terrajet is part of Crossplane organization now! In addition, this release adds many improvements to existing pipelines and unit tests that adds to the maturity of Terrajet.

    Notable Changes

    • Make Terraform resource ID available in status.atProvider by @ulucinar in https://github.com/crossplane/terrajet/pull/160
    • Unit tests for terraform.Workspace by @sergenyalcin in https://github.com/crossplane/terrajet/pull/172
    • Add condition about running async operation by @vaspahomov in https://github.com/crossplane/terrajet/pull/186
    • Configurable operation timeouts by @turkenh in https://github.com/crossplane/terrajet/pull/191

    What's Changed

    • pipeline: use generated parameters name instead of hard-coded by @muvaf in https://github.com/crossplane/terrajet/pull/155
    • Update guide to get tf provider from generate main by @turkenh in https://github.com/crossplane/terrajet/pull/163
    • Make Terraform resource ID available in status.atProvider by @ulucinar in https://github.com/crossplane/terrajet/pull/160
    • typo fix in terraformed template by @muvaf in https://github.com/crossplane/terrajet/pull/165
    • github: remove github actions that are related to publishing container images by @muvaf in https://github.com/crossplane/terrajet/pull/166
    • Directions for calling ./hack/prepare.sh in the provider guide by @ulucinar in https://github.com/crossplane/terrajet/pull/167
    • Instruct to update provider import path in cmd/provider/main.go as well by @turkenh in https://github.com/crossplane/terrajet/pull/168
    • Handle sensitive optionals coming as null by @turkenh in https://github.com/crossplane/terrajet/pull/170
    • Add OWNERS.md in preparation of move to crossplane org by @muvaf in https://github.com/crossplane/terrajet/pull/180
    • Unit tests for terraform.Workspace by @sergenyalcin in https://github.com/crossplane/terrajet/pull/172
    • Capitalize prefix/suffix acronyms in default resource config's kind name by @ulucinar in https://github.com/crossplane/terrajet/pull/179
    • Add more acronyms by @turkenh in https://github.com/crossplane/terrajet/pull/181
    • New acronyms by @muvaf in https://github.com/crossplane/terrajet/pull/183
    • types: add type name to scope earlier by @muvaf in https://github.com/crossplane/terrajet/pull/184
    • go.mod: rename go module to adopt new org in github by @muvaf in https://github.com/crossplane/terrajet/pull/188
    • Assume string type for unset Set/List/Map element types by @ulucinar in https://github.com/crossplane/terrajet/pull/182
    • controller: set external name to its latest value no matter what by @muvaf in https://github.com/crossplane/terrajet/pull/190
    • Add condition about running async operation by @vaspahomov in https://github.com/crossplane/terrajet/pull/186
    • Configurable operation timeouts by @turkenh in https://github.com/crossplane/terrajet/pull/191
    • Empty commit ahead of the 0.3 release by @muvaf in https://github.com/crossplane/terrajet/pull/192

    New Contributors

    • @sergenyalcin made their first contribution in https://github.com/crossplane/terrajet/pull/172
    • @vaspahomov made their first contribution in https://github.com/crossplane/terrajet/pull/186

    Full Changelog: https://github.com/crossplane/terrajet/compare/v0.2.0...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Nov 19, 2021)

    Notable Updates

    Fixed a problem with duplicate parameters type name generated in the same package. See details here: https://github.com/crossplane-contrib/terrajet/pull/156

    What's Changed

    • pipeline: use generated parameters name instead of hard-coded by @github-actions in https://github.com/crossplane-contrib/terrajet/pull/156

    Full Changelog: https://github.com/crossplane-contrib/terrajet/compare/v0.2.0...v0.2.1

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Nov 19, 2021)

    Notable Updates

    The main theme of this release is to stabilize the interfaces for external usage and adding more configuration points to enable even more resources.

    You can now go ahead and generate your own Crossplane provider using Terrajet by following the guide here!

    What's Changed

    • Pass provider credentials as environment variables by @ulucinar in https://github.com/crossplane-contrib/terrajet/pull/111
    • Capture Terraform resource schema version with resource.Terraformed by @ulucinar in https://github.com/crossplane-contrib/terrajet/pull/117
    • external name: use function in the config directly instead of generating the call by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/124
    • controller: return the resource as ready as soon as possible by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/126
    • Add custom config option for defining custom connection keys by @turkenh in https://github.com/crossplane-contrib/terrajet/pull/121
    • Callback for updating status with last operation error by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/103
    • Pipeline abstraction and configuration interface improvements by @turkenh in https://github.com/crossplane-contrib/terrajet/pull/128
    • Do not run configurator chain on a not-included resource by @ulucinar in https://github.com/crossplane-contrib/terrajet/pull/138
    • Find a unique type name by appending indexes by @turkenh in https://github.com/crossplane-contrib/terrajet/pull/137
    • Use explicit root directory so that generator can be run from anywhere by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/145
    • pipeline: put underscore between CRD kind and GVK suffix by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/142
    • pipeline: prepend group and version with CRD so that CRDs whose name is Group can be generated by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/146
    • pipeline: run goimports with absolute path from root dir by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/147
    • Rename Group to ShortGroup and allow empty short group by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/144
    • pipeline: sort processing order, otherwise import lists are unstable by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/148
    • Ability to override ID operations by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/132
    • pipeline: sort import order by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/150
    • Use pointer for secret ref if optional by @turkenh in https://github.com/crossplane-contrib/terrajet/pull/149
    • Extract Terraform error messages from JSON logs by @ulucinar in https://github.com/crossplane-contrib/terrajet/pull/143
    • replace tf strings with jet by @muvaf in https://github.com/crossplane-contrib/terrajet/pull/152
    • Generating a Provider guide by @turkenh in https://github.com/crossplane-contrib/terrajet/pull/130

    Full Changelog: https://github.com/crossplane-contrib/terrajet/compare/v0.1.0...v0.2.0

    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Oct 11, 2021)

    This is the first release of Terrajet!

    You can start using Terrajet today to generate fully XRM-compliant Crossplane providers. Since it's still alpha, there are many things subject to change and we'd love to get your feedback to prioritize the most important ones for Crossplane community.

    Today, there are two providers generated already that you can take a look and give a spin in your clusters:

    Source code(tar.gz)
    Source code(zip)
Owner
Crossplane Contrib
Crossplane Contrib
Provider-template - Template for writing providers for crossplane

provider-template provider-template is a minimal Crossplane Provider that is mea

maros.zeleny 0 Feb 3, 2022
Terraform-equinix-migration-tool - Tool to migrate code from Equinix Metal terraform provider to Equinix terraform provider

Equinix Terraform Provider Migration Tool This tool targets a terraform working

Equinix 1 Feb 15, 2022
provider-kubernetes is a Crossplane Provider that enables deployment and management of arbitrary Kubernetes objects on clusters

provider-kubernetes provider-kubernetes is a Crossplane Provider that enables deployment and management of arbitrary Kubernetes objects on clusters ty

International Business Machines 2 Dec 14, 2022
Provider-milvus - Milvus provider for crossplane

provider-milvus provider-milvus is a minimal Crossplane Provider that is meant t

The Milvus Project 2 Feb 9, 2022
Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)

terraform-provider-awsutils Terraform provider for performing various tasks that cannot be performed with the official AWS Terraform Provider from Has

Cloud Posse 25 Dec 8, 2022
Terraform Provider for Azure (Resource Manager)Terraform Provider for Azure (Resource Manager)

Terraform Provider for Azure (Resource Manager) Version 2.x of the AzureRM Provider requires Terraform 0.12.x and later, but 1.0 is recommended. Terra

null 0 Oct 16, 2021
Terraform-provider-mailcow - Terraform provider for Mailcow

Terraform Provider Scaffolding (Terraform Plugin SDK) This template repository i

Owen Valentine 0 Dec 31, 2021
Terraform-provider-buddy - Terraform Buddy provider For golang

Terraform Provider for Buddy Documentation Requirements Terraform >= 1.0.11 Go >

Buddy 1 Jan 5, 2022
Terraform-provider-vercel - Terraform Vercel Provider With Golang

Vercel Terraform Provider Website: https://www.terraform.io Documentation: https

Vercel 80 Dec 14, 2022
Terraform-provider-age - Age Terraform Provider with golang

Age Terraform Provider This provider lets you generate an Age key pair. Using th

ConsenSys Software 0 Feb 15, 2022
An experimental crossplane provider for @zscaler zpa

provider-zpa Crossplane provider for [Zscaler ZPA] The provider built from this repository can be installed into a Crossplane control plane or run sep

null 0 Dec 7, 2021
Crossplane provider to provision and manage Kubernetes objects on (remote) Kubernetes clusters.

provider-kubernetes provider-kubernetes is a Crossplane Provider that enables deployment and management of arbitrary Kubernetes objects on clusters ty

Crossplane Contrib 69 Jan 3, 2023
Crossplane provider for InfluxDB Cloud

provider-template provider-template is a minimal Crossplane Provider that is meant to be used as a template for implementing new Providers. It comes w

Crossplane Contrib 3 Jan 10, 2022
Crossplane provider for Confluent Cloud

provider-confluent provider-confluent is a minimal Crossplane Provider that is meant to be used as a template for implementing new Providers. It comes

DFDS 1 Feb 4, 2022
A minimal Crossplane Provider For Golang

provider-template provider-template is a minimal Crossplane Provider that is mea

Rajendra Gosavi 0 Dec 19, 2021
kube-champ 43 Oct 19, 2022
Terraform-in-Terraform: Execute Modules directly from the Terraform Registry

Terraform-In-Terraform Provider This provider allows running Terraform in Terraform. This might seem insane but there are some edge cases where it com

WeakPixel 39 Dec 25, 2022
Terraform utility provider for constructing bash scripts that use data from a Terraform module

Terraform Bash Provider This is a Terraform utility provider which aims to robustly generate Bash scripts which refer to data that originated in Terra

Martin Atkins 33 Sep 6, 2022
Quick start repository for creating a Terraform provider using terraform-plugin-framework

Terraform Provider Scaffolding (Terraform Plugin Framework) This template repository is built on the Terraform Plugin Framework. The template reposito

HashiCorp 70 Dec 15, 2022