Repository navigation
Releases
Releases · dataform-co/dataform
3.1.5
What's Changed
- docs: add BigQuery property graphs user guide (docs/property_graphs.md) by @rafal-hawrylak in #2422
- feat(cli): generate the CLI reference tables in cli/README.md from the option definitions by @mbiernacik in #2419
- feat(cli)!: allow --json to be used without --dry-run in run by @Ford544 in #2418
- feat(cli): make location optional in init and init-creds by @Ford544 in #2402
- Add warning if set @@dataset_id used with no defaultLocation by @Ford544 in #2421
- fix(cli): attach ClientInfo gRPC metadata header to lineage run events by @rafal-hawrylak in #2425
- chore(cli): address review comments from #2425 on lineage ClientInfo header by @rafal-hawrylak in #2429
- Bump DF_VERSION from 3.1.4 to 3.1.5 by @rafal-hawrylak in #2430
Full Changelog: 3.1.4...3.1.5
3.1.4
What's Changed
- feat(cli): create schemas lazily by @Ford544 in #2408
- JiT SQLX support by @ikholopov-omni in #2087
- Filter stateless-install project copies with ignore-file overrides by @dmargol1 in #2272
- Adapt PropertyGraph graph.yaml schema and DDL compiler to Semantic Ontology V1 by @rafal-hawrylak in #2417
- docs(core): document PropertyGraphConfig in configs reference by @rafal-hawrylak in #2420
- Bump DF_VERSION from 3.1.3 to 3.1.4 by @rafal-hawrylak in #2423
New Contributors
Full Changelog: 3.1.3...3.1.4
3.1.3
What's Changed
- test(api): use explicit unknownProperty in credentials unknown-field test by @SergikL in #2398
- fix(cli): use the pinned @dataform/core for JiT compilation by @mbiernacik in #2392
- Pass JiT compiler arguments as an options object by @mbiernacik in #2400
- fix(cli): emit lineage inputs/outputs on COMPLETE for actions with a BigQuery output and use 3-part actionName by @rafal-hawrylak in #2399
- docs(core): fix Operation JS API example to use
queries()and config arg by @apilaskowski in #2409 - Prefetch npm dependencies for Dependabot PRs by @apilaskowski in #2405
- fix(core): require defaultDataset for incremental Lakehouse tables by @SergikL in #2406
- docs(cli): add user guide and config reference for data lineage emission by @rafal-hawrylak in #2411
- fix(cli): disable lineage emitter and display a warning if location is not set by @Ford544 in #2393
- docs(cli): specify roles/datalineage.producer and processOpenLineageMessage permission by @rafal-hawrylak in #2412
- support defaultJobLabels and action-level jobLabels by @zaptot in #2344
- Update markdown-it to 14.3.3 by @Ford544 in #2410
- fix(cli): concatenateQueries comment issues by @Ford544 in #2414
- feat(cli)!: concatenate operations queries into a single task by @Ford544 in #2413
- Publishing Dataform security patches by @dataform-team in #2415
Full Changelog: 3.1.2...3.1.3
3.1.2
What's Changed
- fix(cli): allow BigQuery to determine location if unset by @Ford544 in #2396
- refactor(core): rename property graph entities/relationships to nodes/edges by @rafal-hawrylak in #2387
- Bump DF_VERSION from 3.1.1 to 3.1.2 by @rafal-hawrylak in #2397
Full Changelog: 3.1.1...3.1.2
3.1.1
What's Changed
- refactor(cli): migrate BigQuery integration tests from KMS-encrypted test key to ADC by @mbiernacik in #2362
- fix(cli): filter out non-existent tables in BigQueryDbAdapter.search by @rafal-hawrylak in #2388
- test: Make bigquery search integration test ignore leftover datasets by @apilaskowski in #2390
- fix(tests): Ensure concurrent test runs do not operate on the same BQ datasets by @Ford544 in #2385
- feat(core): add support for 4-part Lakehouse Catalog targets by @SergikL in #2384
- fix(cli): Use defaultLocation for BigQuery jobs and datasets in dataform run by @apilaskowski in #2389
- Canary test (with GCP bucket) by @Ford544 in #2354
- fix(cli): remove unused sql and errorMessage OpenLineage facets by @rafal-hawrylak in #2391
- test(cli): add location resolution e2e tests by @Ford544 in #2394
- Bump DF_VERSION from 3.1.0 to 3.1.1 by @SergikL in #2395
New Contributors
Full Changelog: 3.1.0...3.1.1
3.1.0
What's Changed
- Bump fast-uri from 3.1.4 to 3.1.6 by @dependabot[bot] in #2270
- Update KMS test credentials by @rafal-hawrylak in #2346
- Pin fast-uri to 3.1.7 by @Ford544 in #2349
- Publishing Dataform security patches by @dataform-team in #2353
- Bump @grpc/grpc-js from 1.14.4 to 1.14.5 by @dependabot[bot] in #2355
- Bump vm2 from 3.11.6 to 3.11.7 by @dependabot[bot] in #2358
- Bump vm2 from 3.11.7 to 3.12.2 by @dependabot[bot] in #2363
- docs(cli): add comprehensive CLI reference README (#2329) by @rafal-hawrylak in #2364
- Bazel toolchain rewrite by @Ford544 in #2352
- Add a binary run test by @Ford544 in #2356
- Update TypeScript, ts-eslint and rollup by @Ford544 in #2365
- refactor(cli): replace deprecated vm2 with native node:vm runner by @mbiernacik in #2310
- feat(cli): add Windows support and automated CI test coverage by @mbiernacik in #2350
- Update pnpm to 9.7.1 by @Ford544 in #2366
- Bump baseline-browser-mapping from 2.9.19 to 2.11.25 by @apilaskowski in #2370
- Bump browserslist from 4.28.1 to 4.29.0 by @apilaskowski in #2371
- feat(cli): emit lineage for assertions and property graphs by @rafal-hawrylak in #2368
- fix(cli): stop rejecting older @dataform/core versions after a CLI version bump by @mbiernacik in #2376
- Add proto fields for defaultJobLabels and action-level jobLabels by @zaptot in #2351
- Upgrade chokidar to v4 to drop vulnerable braces dependency by @apilaskowski in #2377
- chore(cli): address review comments from PR #2368 by @rafal-hawrylak in #2373
- Bump DF_VERSION from 3.0.72 to 3.1.0 by @rafal-hawrylak in #2375
- Bump serialize-javascript override to ^7.1.2 by @apilaskowski in #2379
- docs(tooling): remove outdated line from contributing.md by @Ford544 in #2381
- Bump brace-expansion override to 5.0.12 by @apilaskowski in #2378
- fix(tooling): fix publish script by @Ford544 in #2380
- test(core): cover lineage settings in workflow_settings.yaml, including an empty lineage block by @rafal-hawrylak in #2382
Full Changelog: 3.0.71...3.1.0
3.0.71
What's Changed
- fix(security): upgrade linkify-it to 5.0.2 by @mbiernacik in #2324
- Update minimatch by @Ford544 in #2325
- Remove rimraf by @Ford544 in #2326
- chore(deps): upgrade braces to 3.0.3 by @mbiernacik in #2327
- chore(deps): upgrade serialize-javascript to 7.1.1 by @mbiernacik in #2331
- Update parse-duration by @Ford544 in #2332
- Minimatch update by @Ford544 in #2330
- Pin uuid version to 1.14.2 by @Ford544 in #2333
- Update markdown-it by @Ford544 in #2335
- chore(deps): remove unused ts-loader dependency by @mbiernacik in #2336
- Update debug by @Ford544 in #2337
- Split devDependencies off dependencies by @Ford544 in #2338
- hoist DECLARE statements to start of onSchemaChange procedure (#2307) by @apilaskowski in #2319
- Remove unused git workflow by @Ford544 in #2341
- Format entire repo with prettier by @rafal-hawrylak in #2342
- Sqlx comment parsing backslashes by @Ford544 in #2339
- chore(core): remove unused extractJsBlocks to resolve CodeQL ReDoS alert by @mbiernacik in #2340
- Bump DF_VERSION from 3.0.70 to 3.0.71 by @apilaskowski in #2334
Full Changelog: 3.0.70...3.0.71
3.0.70
What's Changed
- refactor(cli): decompose monolithic cli/index.ts into modular command modules by @mbiernacik in #2286
- Eliminate hardcoded project names for integration tests by @Ford544 in #2285
- Mark tests that require valid GCP project credentials as integration tests by @Ford544 in #2289
- refactor(cli): clean up help command handling and encapsulate yargs configuration by @mbiernacik in #2290
- refactor(cli): scope quiet compilation option to compile command by @mbiernacik in #2291
- Refactor cli tests by @Ford544 in #2287
- build(bazel): update Go modules versions and speed up local run script by @mbiernacik in #2293
- Split tests/integration/bigquery.spec.ts by @Ford544 in #2296
- Split the large test in tests/api/projects.spec.ts by @Ford544 in #2295
- Split tests/api/api.spec.ts by @Ford544 in #2294
- chore(lint): migrate from tslint to eslint by @mbiernacik in #2297
- Split core/main_test.ts by @Ford544 in #2301
- Split core/actions/table_test and core/actions/incremental_table_test by @Ford544 in #2302
- Split core/actions/assertion_test.ts by @Ford544 in #2305
- refactor(cli): strongly type yargs commands by @mbiernacik in #2306
- Refactor core/main_property_graphs_test.ts by @Ford544 in #2304
- Mitigate core/index_compile_test timeouts by @Ford544 in #2308
- Remove redundant request dependency by @Ford544 in #2311
- Bump js-yaml from 4.3.0 to 4.3.2 by @dependabot[bot] in #2299
- chore(deps): remove typedoc dependency to resolve marked vulnerability by @mbiernacik in #2313
- Upgrade prettier to 3.9.4 by @Ford544 in #2315
- fix: cli compile time increased by ~4x by @ashish10alex in #2300
- Upgrade jsdoc to ^4.0.5 by @Ford544 in #2316
- Update form-data to 2.5.6 by @Ford544 in #2317
- Enforce prettier formatting standards on PRs by @Ford544 in #2312
- Preserve Governance Controls in Incremental Tables. by @mohamed-saad in #2314
- Bump DF_VERSION from 3.0.69 to 3.0.70 by @mohamed-saad in #2318
New Contributors
- @mbiernacik made their first contribution in #2286
- @Ford544 made their first contribution in #2285
Full Changelog: 3.0.69...3.0.70
3.0.69
🕸️ BigQuery Graph Integration (Property Graphs)
With this release, Dataform CLI delivers full support and native integration for BigQuery Graph (Property Graphs). You can now declaratively define, orchestrate, and manage BigQuery Property Graphs directly in your Dataform projects via YAML.
Key Integration Capabilities:
- Declarative YAML Definitions: Define property graph schemas using
definitions/graph.yamlto specify node tables (entities), edge tables (relationships), source/destination endpoints, labels, and properties. - Automated Node & Edge DAG Dependencies: Reference upstream Dataform models in entities/relationships using
refdata sources (dataSourceRef), automatically resolving dataset targets and ensuring correct execution ordering after underlying tables. - Tagging & Selective Execution: Full
tagssupport ingraph.yamlto filter and run property graphs during selective executions (e.g.,dataform run --tags graph). - Assertion Propagation Control: Explicitly control whether dependent assertions run before property graph creation (
dependOnDependencyAssertions: false/includeDependentAssertions: false). - Automated BigQuery DDL Lifecycle: Compiles declarative YAML definitions into BigQuery
CREATE OR REPLACE PROPERTY GRAPHDDL statements and executes them in BigQuery.
What's Changed
- Fix View.verifyConfig mutating caller-shared bigquery configs by @rafal-hawrylak in #2273
- Support tags on PropertyGraph actions by @rafal-hawrylak in #2275
- Bump fast-xml-parser from 5.5.6 to 5.7.3 by @dependabot[bot] in #2163
- Preserve explicit includeDependentAssertions=false on PropertyGraph refs by @rafal-hawrylak in #2281
- Fix Session.declare mutating caller-shared configs (#2280) by @rafal-hawrylak in #2283
- Bump DF_VERSION from 3.0.68 to 3.0.69 by @rafal-hawrylak in #2284
Full Changelog: 3.0.68...3.0.69
3.0.68
What's Changed
- Support ref() and DAG dependencies in PropertyGraph by @rafal-hawrylak in #2258
- Fix plumb filename by @rafal-hawrylak in #2271
- Add CLI runtime coverage and E2E integration test for PropertyGraph by @rafal-hawrylak in #2268
- Bump DF_VERSION from 3.0.67 to 3.0.68 by @rafal-hawrylak in #2276
Full Changelog: 3.0.67...3.0.68