Skip to content

feat: support postgresql numeric column type - #1311

Merged
k1LoW merged 1 commit into
k1LoW:mainfrom
greenteasocha:feature/postgresql-numeric
Oct 4, 2025
Merged

k1LoW merged 1 commit into
k1LoW:mainfrom
greenteasocha:feature/postgresql-numeric

Conversation

@greenteasocha

@greenteasocha greenteasocha commented Oct 4, 2025 •

Copy link
Copy Markdown
Contributor

Hello, thank you for maintaining such a great tool.

Recently, I encountered a situation that I want to handle postgresql NUMERIC type column.
This PR will fix this error.

 Failure/Error: db query failed on "Test fixed-point data types".steps[0]: invalid column: evaluated n_decimal, but got NUMERIC(1234.56789): strconv.Atoi: parsing "1234.56789": invalid syntax

Suggested Feature

Support postgresql NUMERIC column type. It also support DECIMAL, which is aliased to NUMERIC.

Minimal reproducible example

init.sql

CREATE TABLE IF NOT EXISTS "runn_test" (
    n_decimal DECIMAL(10,5) NOT NULL,
    n_numeric NUMERIC(10,5) NOT NULL
);

INSERT INTO "runn_test" (n_decimal, n_numeric) VALUES (1234.56789, 98765.4321);

compose.yml

services:
  postgres:
    image: postgres:17-alpine
    container_name: runn-postgres
    environment:
      POSTGRES_DB: runn_test
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
    ports:
      - "127.0.0.1:5432:5432"
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql

test_postgres.yml

desc: Test fixed-point data types
runners:
  db:
    dsn: postgres://postgres:password@localhost:5432/runn_test?sslmode=disable

steps:
  - db:
      query: SELECT * FROM runn_test
    test: |
      current.rows[0].n_decimal == 1234.56789
      && current.rows[0].n_numeric == 98765.4321

Backward compatibility check for other supported DBs

compose.yml (init_mysql.sql is shown in following section)

services:
  mysql:
    image: mysql:8.0
    container_name: runn-mysql
    environment:
      MYSQL_DATABASE: runn_test
      MYSQL_ROOT_PASSWORD: rootpassword
      MYSQL_USER: mysql
      MYSQL_PASSWORD: password
    ports:
      - "127.0.0.1:3306:3306"
    volumes:
      - ./init_mysql.sql:/docker-entrypoint-initdb.d/init.sql

  sqlite:
    image: alpine:3.22
    container_name: runn-sqlite
  
    command: sh -c "apk add --no-cache sqlite=3.49.2-r1 && sleep infinity"
    volumes:
      - ./data:/app/data

  spanner:
    image: gcr.io/cloud-spanner-emulator/emulator
    container_name: runn-spanner
    ports:
      - "9010:9010"
      - "9020:9020"

MySQL

init_mysql.sql

CREATE TABLE IF NOT EXISTS `runn_test` (
    n_decimal DECIMAL(10,5) NOT NULL,
    n_numeric NUMERIC(10,5) NOT NULL
);

INSERT INTO `runn_test` (n_decimal, n_numeric) VALUES (1234.56789, 98765.4321);

test_mysql.yml

desc: Test fixed-point data types (MySQL)
runners:
  db:
    dsn: mysql://root:rootpassword@localhost:3306/runn_test

steps:
  - db:
      query: SELECT * FROM `runn_test`
    test: |
      current.rows[0].n_decimal == 1234.56789
      && current.rows[0].n_numeric == 98765.4321

SQLite3

init_sqlite.sql

CREATE TABLE users (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  n NUMERIC
);

INSERT INTO users (n) VALUES (1234.56789);
docker-compose exec -T sqlite sqlite3 /app/data/database.db < init_sqlite.sql

test_sqlite.yml

desc: Test fixed-point data types (SQLite3)
runners:
  db:
    dsn: sqlite://./data/database.db
steps:
  -
    desc: Verify both records exist with correct NUMERIC values
    db:
      query: |
        SELECT id, n FROM users ORDER BY id;
    test: |
      current.rows[0].n == 1234.56789
  

Cloud Spanner

gcloud config configurations create runn-emulator
gcloud config set auth/disable_credentials true
gcloud config set project runn-test-project
gcloud config set api_endpoint_overrides/spanner http://localhost:9020/

gcloud spanner instances create runn-test-instance --config=emulator-config --description="Test Instance" --nodes=1
gcloud spanner databases create test-db  --instance=runn-test-instance    
gcloud spanner databases ddl update test-db --instance=runn-test-instance --ddl="CREATE TABLE numeric_test (n NUMERIC NOT NULL) PRIMARY KEY (n);" 
gcloud spanner rows insert --instance=runn-test-instance --database=test-db --table=numeric_test --data=n=1234.56789

test_spanner.yml

(run with envirionment variable set. SPANNER_EMULATOR_HOST=localhost:9010 runn run test_spanner.yml)

desc: Test fixed-point data types (Cloud Spanner)
runners:
  db:
    dsn: spanner://runn-test-project/runn-test-instance/test-db
steps:  
# Unfortunately, I couldn't figure out how to test its value directly, but no error occurs when handling query results
  - db:
      query: SELECT n FROM numeric_test
# possible workaround
  - db:
      query: SELECT CAST(n AS STRING) as n_string FROM numeric_test
    test: current.rows[0].n_string == "1234.56789"

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @greenteasocha, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces crucial support for the PostgreSQL NUMERIC column type, which was previously causing parsing errors when attempting to convert fixed-point numbers. By updating the database runner's type handling logic, NUMERIC and DECIMAL values are now correctly interpreted as floating-point numbers, enhancing compatibility and preventing data retrieval failures across various SQL databases, including PostgreSQL and MySQL.

Highlights

  • PostgreSQL NUMERIC Support: Added support for PostgreSQL's NUMERIC column type, resolving an strconv.Atoi parsing error for fixed-point numbers.
  • Unified Fixed-Point Handling: The change unifies the handling of DECIMAL and NUMERIC types across different database systems (e.g., PostgreSQL and MySQL) by parsing them as floating-point numbers.
  • Code Logic Update: Modified the db.go file to explicitly include NUMERIC in the list of types that are converted using strconv.ParseFloat.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for the PostgreSQL NUMERIC data type by treating it similarly to DECIMAL. The change is correct and fixes the issue described. My review includes two main points. First, a high-severity concern about potential precision loss when converting NUMERIC and DECIMAL types to float64, which could lead to incorrect behavior in tests. I've recommended creating a follow-up task to address this. Second, I've suggested adding an automated unit test to ensure the long-term stability of this new feature. Overall, this is a good contribution that addresses a real-world use case.

Comment thread db.go
case t == "DECIMAL" || t == "FLOAT" || t == "DOUBLE": // MySQL: NUMERIC = DECIMAL ( FLOAT, DOUBLE maybe not used )
// MySQL: NUMERIC is aliased to DECIMAL. PostgreSQL: DECIMAL is aliased to NUMERIC. ( FLOAT, DOUBLE maybe not used )
case t == "DECIMAL" || t == "NUMERIC" || t == "FLOAT" || t == "DOUBLE":
num, err := strconv.ParseFloat(s, 64) //nostyle:repetition

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using strconv.ParseFloat for DECIMAL and NUMERIC types can cause precision loss, as float64 cannot accurately represent all exact decimal values. This can lead to incorrect test evaluations, especially with financial data.

A safer approach would be to either keep these values as strings or use a high-precision decimal library. Since this change is consistent with how DECIMAL is already handled, it's acceptable for now, but I strongly suggest creating a follow-up issue to address this for both DECIMAL and NUMERIC types to prevent potential data-related bugs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This precision issue already exists for DECIMAL types in the current implementation. This PR maintains that existing behavior and simply extends it to NUMERIC.

Comment thread db.go
row[c] = s
case t == "DECIMAL" || t == "FLOAT" || t == "DOUBLE": // MySQL: NUMERIC = DECIMAL ( FLOAT, DOUBLE maybe not used )
// MySQL: NUMERIC is aliased to DECIMAL. PostgreSQL: DECIMAL is aliased to NUMERIC. ( FLOAT, DOUBLE maybe not used )
case t == "DECIMAL" || t == "NUMERIC" || t == "FLOAT" || t == "DOUBLE":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The manual testing in the PR description is excellent. To prevent future regressions, please consider adding an automated unit test to db_test.go. A new test case within TestDBRunner that uses a NUMERIC column would be a valuable addition to solidify this feature.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at db_test.go, it seems that the tests primarily target SQLite3 rather than PostgreSQL, which is what this change addresses.

Additionally, the existing tests already include queries against NUMERIC type columns (e.g., created_at), so NUMERIC handling is already covered to some extent.

I'll keep the current implementation as is.
But also, I want to hear maintainers' opinion.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine as is in the current test environment.

@k1LoW k1LoW left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@greenteasocha Thank you for your contribution!

@k1LoW
k1LoW merged commit 990eed2 into k1LoW:main Oct 4, 2025
@github-actions github-actions Bot mentioned this pull request Sep 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

⌂ 换网址