Skip to content
This repository was archived by the owner on Oct 20, 2026. It is now read-only.
This repository was archived by the owner on Oct 20, 2026. It is now read-only.

runStream returning incomplete and out of bounds for non existing values #1959

Description

@saranyasengo

Environment details

  • OS: Google Container OS
  • Node.js version: 18.15
  • npm version: 9.5.0
  • @google-cloud/spanner version: 5.18.0

Steps to reproduce

This error is inconsistently happening its similar to issues #197 and #180

The code which gives error
const rowsStream = spannerDb.runStream({ sql: SELECT * FROM ${TABLE_NAME} ORDER BY storeId, started DESC, json: true, });

Error thrown as below

GoogleError: Serializing column "started" encountered an error: Integer 211959131678161148 is out of bounds. Call row.toJSON({ wrapNumbers: true }) to receive a custom type. at Int.valueOf (/opt/app/node_modules/@google-cloud/spanner/build/src/codec.js:121:19) at convertValueToJson (/opt/app/node_modules/@google-cloud/spanner/build/src/codec.js:263:22) at Object.convertFieldsToJson (/opt/app/node_modules/@google-cloud/spanner/build/src/codec.js:239:31) at Array.value (/opt/app/node_modules/@google-cloud/spanner/build/src/partial-result-stream.js:206:38) at PartialResultStream._addValue (/opt/app/node_modules/@google-cloud/spanner/build/src/partial-result-stream.js:187:34) at /opt/app/node_modules/@google-cloud/spanner/build/src/partial-result-stream.js:163:24 at Array.forEach (<anonymous>) at PartialResultStream._addChunk (/opt/app/node_modules/@google-cloud/spanner/build/src/partial-result-stream.js:162:16) at PartialResultStream._transform (/opt/app/node_modules/@google-cloud/spanner/build/src/partial-result-stream.js:85:24) at Transform._write (node:internal/streams/transform:175:8) at obj.<computed> [as _write] (/opt/app/node_modules/stubs/index.js:26:26) at doWrite (node:internal/streams/writable:411:12) at clearBuffer (node:internal/streams/writable:572:7) at onwrite (node:internal/streams/writable:464:7) at node:internal/streams/transform:190:7 at PartialResultStream._tryResume (/opt/app/node_modules/@google-cloud/spanner/build/src/partial-result-stream.js:108:13)

started column in table doesn't has the value '211959131678161148' thrown. The storeId column has this value. Looks like it swapped or wrongly mapping the values for columns. 

This error not thrown always and not reproducible in local as well.

Thanks!

Activity

  1. added
    priority: p2Moderately-important priority. Fix may not be included in next release.
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    on Nov 23, 2023
  2. self-assigned this
    on Dec 11, 2023
  3. saranyasengo commented on Feb 6, 2024

    @saranyasengo
    Author

    @surbhigarg92 Can we get any update on this. spanner runStream causing the issues with swapping the column values

  4. surbhigarg92 commented on Feb 8, 2024

    @surbhigarg92
    Contributor

    Hi @saranyasengo Sorry for the delay. I haven't been able to look into this issue. Can you please share the below information to help me reproduce this issue ?

    1. Did you start seeing this issue recently or was it always there ?
    2. Can you share the schema of your table ?
    3. What is the size of the table ?
    4. Can you try upgrading to the latest version of library and see if you are able to reproduce this issue ?
  5. saranyasengo commented on Feb 13, 2024

    @saranyasengo
    Author

    Hey @surbhigarg92

    I believe the issue is from so long, but we captured recently since its thrown out of bounds error. But the actual issue is column values are swapped.

    We have upgraded recently to @google-cloud/spanner version 7.0.0 still we see the values are getting swapped.

    Schema of the table is secure to share. We have created google support ticket for this. I will share the table schema, sample data's with table and how the spanner runStream returns the data in support ticket.

  6. added 2 commits that reference this issue on Feb 23, 2024
    f337089
    53e529f
  7. surbhigarg92 commented on Feb 23, 2024

    @surbhigarg92
    Contributor

    @saranyasengo Can you please try using the latest release https://github.com/googleapis/nodejs-spanner/releases/tag/v7.4.0 and let us know if this resolves your issue ?

  8. saranyasengo commented on Feb 23, 2024

    @saranyasengo
    Author

    @surbhigarg92 I will update to latest version and since its not reproducible in local and its coming weekly once. I will update and monitor for couple of weeks and update on this.

    Can you please share us the reproduction step to confirm before using the latest one?

  9. surbhigarg92 commented on Feb 24, 2024

    @surbhigarg92
    Contributor

    We found that error was happening in a particular case where the pending value were getting cleared when partialResultSet had a resume token . Please refer #2000 for the fix and test cases added.

  10. saranyasengo commented on Feb 28, 2024

    @saranyasengo
    Author

    @surbhigarg92 I have upgraded the version to 7.4.0 and now I see this issue is not coming for past 2 days, But we are facing different type of issue now, hence rollback the changes to previous one.

    The specific error we are getting is Error: 3 INVALID_ARGUMENT: The code field is required for types. and I will attach the table schema and query details in the google support ticket. Please help us on fixing this.

  11. surbhigarg92 commented on Mar 1, 2024

    @surbhigarg92
    Contributor

    @saranyasengo The error looks like because of a recent feature which was launched #1869

    I am working on fixing this. Sorry for the inconvenience.

  12. surbhigarg92 commented on Mar 4, 2024

    @surbhigarg92
    Contributor

    @saranyasengo As communicated earlier the error was happening because of a recent feature launch. We have reverted that feature for the time being.

    Can you please try using 7.5.0 release

  13. saranyasengo commented on Mar 5, 2024

    @saranyasengo
    Author

    @surbhigarg92 We will use 7.5.0 and will monitor for somedays for fixes.

  14. surbhigarg92 commented on Mar 18, 2024

    @surbhigarg92
    Contributor

    @saranyasengo Let us know if this worked for you

  15. saranyasengo commented on Mar 26, 2024

    @saranyasengo
    Author

    @surbhigarg92 it works, So far we have not faced this issue.

  16. surbhigarg92 commented on Mar 27, 2024

    @surbhigarg92
    Contributor

    @saranyasengo We are closing this ticket. Please feel free to reopen if you face any issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: spannerIssues related to the googleapis/nodejs-spanner API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    ⌂ 换网址