Skip to content

Bigtables Garbage Collection Policy is not applicable #300

Closed

Description

Describe the bug
When applying a simple BigtableGCPolicy like the one here in the documentation, the objects are created properly, but the policy is not really applied.

image

If the policy is updated using the cli cbt it works fine

ConfigConnector Version
1.27.1

To Reproduce
Steps to reproduce the behavior:

YAML snippets:

apiVersion: bigtable.cnrm.cloud.google.com/v1beta1
kind: BigtableGCPolicy
metadata:
  name: bigtablegcpolicy-sample
spec:
  tableRef:
    name: bigtablename
  columnFamily: quote
  instanceRef:
    name: clustername
  mode: INTERSECTION
  maxAge:
  - days: 30
  maxVersion:
  - number: 10

Then check the GC policies on the field using cbt command line, it will show <never>

If the policy is update manually cbt setgcpolicy ... it will work fine

Debugging:
kubectl describe BigtableGCPolicy -n datastores-wes-sit

Name:         datastores-wes-sit-trade-gcp
Namespace:    datastores-wes-sit
Labels:       <none>
Annotations:  cnrm.cloud.google.com/management-conflict-prevention-policy: none
              cnrm.cloud.google.com/project-id: qt-msa-nonprod-6f
              kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"bigtable.cnrm.cloud.google.com/v1beta1","kind":"BigtableGCPolicy","metadata":{"annotations":{},"name":"datastores-wes-sit-t...
API Version:  bigtable.cnrm.cloud.google.com/v1beta1
Kind:         BigtableGCPolicy
Metadata:
  Creation Timestamp:  2020-11-05T19:11:11Z
  Finalizers:
    cnrm.cloud.google.com/finalizer
    cnrm.cloud.google.com/deletion-defender
  Generation:  2
  Managed Fields:
.....................
.....................
    Manager:         cnrm-controller-manager
    Operation:       Update
    Time:            2020-11-05T19:11:11Z
  Resource Version:  168755889
  Self Link:         /apis/bigtable.cnrm.cloud.google.com/v1beta1/namespaces/datastores-wes-sit/bigtablegcpolicies/datastores-wes-sit-trade-gcp
  UID:               ffd50825-f3e6-43da-b048-6e3fbb956019
Spec:
  Column Family:  trade
  Instance Ref:
    Name:  shared-nonprod
  Max Age:
    Days:  10
  Max Version:
    Number:  10
  Mode:      INTERSECTION
  Table Ref:
    Name:  datastores-wes-sit
Status:
  Conditions:
    Last Transition Time:  2020-11-05T19:11:11Z
    Message:               The resource is up to date
    Reason:                UpToDate
    Status:                True
    Type:                  Ready
Events:
  Type    Reason    Age   From                         Message
  ----    ------    ----  ----                         -------
  Normal  UpToDate  94s   bigtablegcpolicy-controller  The resource is up to date

Activity

  1. jcanseco commented on Nov 10, 2020

    @jcanseco
    Member

    Hi @wesam80, thanks, I've confirmed that this is a bug on our side. We'll work on fixing the issue and let you know when we have an update.

  2. jcanseco commented on Nov 11, 2020

    @jcanseco
    Member

    Hi @wesam80, I believe we theoretically should be able to support the creation case by late Nov or early Dec (i.e. the case of setting a GC policy when there is none).

    However, it'll take us awhile longer to support the update case (i.e. the case of updating a GC policy that is already there). In theory, you should still be able to perform an "update" once we fix the creation case, but you'd have to do so by deleting and then re-creating the GC policy.

    The problem behind this issue is unfortunately a bit tricky, so these are the best estimates I can give you for now.

    What do you think? Would the fix for the creation case be sufficient for your use-case, or would you rather just wait for both the creation and update cases to be fixed given your production timelines?

  3. jcanseco commented on Nov 12, 2020

    @jcanseco
    Member

    Great, thanks for confirming @wesam80. We'll let you know once we have more updates.

  4. maqiuyujoyce commented on Dec 2, 2020

    @maqiuyujoyce
    Collaborator

    Hi @wesam80 , we just released Config Connector v1.32.0, which fixes the creation case for BigtableGCPolicy. Please let us know if it works.

  5. maqiuyujoyce commented on Dec 4, 2020

    @maqiuyujoyce
    Collaborator

    Hi @wesam80 , thank you for confirming that the creation works!

    In regard to supporting updates, currently we don't have a timeline yet. It's a bigger change compared to supporting creations, so we are still looking into how to better address it. Will let you know when we have any update.

  6. kevinsi4508 commented on Oct 25, 2022

    @kevinsi4508

    This should be resolved by using gc_rules. Currently we are waiting for KCC to update to the latest Google Terraform Provider version.

  7. mbzomowski commented on Nov 17, 2022

    @mbzomowski

    spec.gcRules has been added to BigTableGCPolicy as of Config Connector v1.97.0. We don't recommend using maxAge, maxVersion or mode for defining a
    BigTableGCPolicy as these fields have known drift detection issues. Please use gcRules instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      ⌂ 换网址