Skip to content

fix(datastore): Ignore field mismatch errors - #8694

Merged
bhshkh merged 9 commits into
googleapis:mainfrom
bhshkh:fix/issue-913
Jul 30, 2024
Merged

bhshkh merged 9 commits into
googleapis:mainfrom
bhshkh:fix/issue-913

Conversation

@bhshkh

@bhshkh bhshkh commented Oct 12, 2023 •

Copy link
Copy Markdown
Contributor

ErrFieldMismatch is returned when a field is to be loaded into a different type than the one it was stored from, or when a field is missing or unexported in the destination struct.

This PR allows user to suppress such errors. Some scenarios in which this would be useful are listed in #913

Points to note

  1. Suppressing the error does not mean that Datastore will not fetch missing field. It just means that the field was still fetched from Datastore service but not loaded in user's struct.
  2. When writing the data back to Datastore, it will be overwritten. E.g.
    If entity written to Datastore is {X: 1, Y:2} and it is loaded into type NewStruct struct{X int}, then {X:1} is returned to the user. Now, if user modifies this X = 5 and writes it back to Datastore, there will be no Y field left for this entity in Datastore.

Fixes: #913

@bhshkh
bhshkh requested review from a team October 12, 2023 08:53
@product-auto-label product-auto-label Bot added size: m Pull request size is medium. api: datastore Issues related to the Datastore API. labels Oct 12, 2023
@bhshkh
bhshkh requested a review from enocom October 12, 2023 08:53
Comment thread datastore/datastore.go Outdated
// IgnoreFieldMismatch allows ignoring ErrFieldMismatch error while
// reading or querying data.
// WARNING: Ignoring ErrFieldMismatch can cause data loss
func (c *Client) IgnoreFieldMismatch() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider expressing this as a client specific ClientIOption. Here is an example from storage:

func WithXMLReads() option.ClientOption {
return &withReadAPI{useJSON: false}
}
type withReadAPI struct {
internaloption.EmbeddableAdapter
useJSON bool
}
func (w *withReadAPI) ApplyStorageOpt(c *storageConfig) {
c.useJSONforReads = w.useJSON
c.readAPIWasSet = true
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the PR

@product-auto-label product-auto-label Bot added the stale: old Pull request is old and needs attention. label Nov 11, 2023
@product-auto-label product-auto-label Bot added stale: extraold Pull request is critically old and needs prioritization. and removed stale: old Pull request is old and needs attention. labels Dec 11, 2023
@bhshkh
bhshkh requested a review from a team April 16, 2024 19:04
@bhshkh
bhshkh requested a review from gkevinzheng July 22, 2024 23:04
@bhshkh
bhshkh enabled auto-merge (squash) July 23, 2024 19:38
Comment thread datastore/option.go
// WithIgnoreFieldMismatch allows ignoring ErrFieldMismatch error while
// reading or querying data.
// WARNING: Ignoring ErrFieldMismatch can cause data loss
func WithIgnoreFieldMismatch() option.ClientOption {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Is it better to call this type/field IgnoreFieldMismatch without the "with"?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enocom
enocom removed their request for review July 24, 2024 18:36
@codyoss

codyoss commented Jul 24, 2024

Copy link
Copy Markdown
Member

@bhshkh do we still want this PR?

@bhshkh

bhshkh commented Jul 26, 2024

Copy link
Copy Markdown
Contributor Author

@bhshkh do we still want this PR?

Yes

@bhshkh
bhshkh merged commit 6625d12 into googleapis:main Jul 30, 2024
@bhshkh
bhshkh deleted the fix/issue-913 branch July 30, 2024 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: datastore Issues related to the Datastore API. size: m Pull request size is medium. stale: extraold Pull request is critically old and needs prioritization.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

datastore: When loading kinds into structs, ignore items where no struct field exist instead of generating an error

3 participants

⌂ 换网址