Repository navigation
Permalink
Choose a base ref
{{ refName }}
default
Choose a head ref
{{ refName }}
default
Comparing changes
Choose two branches to see what’s changed or to start a new pull request.
If you need to, you can also or
learn more about diff comparisons.
Open a pull request
Create a new pull request by comparing changes across two branches. If you need to, you can also .
Learn more about diff comparisons here.
base repository: dataform-co/dataform
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 3.1.3
Could not load branches
Nothing to show
Loading
Could not load tags
Nothing to show
{{ refName }}
default
Loading
...
head repository: dataform-co/dataform
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 3.1.4
Could not load branches
Nothing to show
Loading
Could not load tags
Nothing to show
{{ refName }}
default
Loading
- 6 commits
- 58 files changed
- 4 contributors
Commits on Nov 3, 2026
-
feat(cli): create schemas lazily (#2408)
* Create schemas lazily if location unset * Wrap all tasks in an action * Remove old path; always create schemas lazily * Add lock on schema creation * Dry-run select statements in wrapped scripts * Include pre- and postops in dry run validation query * Do not wait for dataset creation if task skipped * Fixes * Split off schema creation tests into a seperate file * Split location tests to a seperate file * Adjust error position * Reformat * Only create operations schemas if hasOutput * Rebase fix * Mark actions as cancelled when cancel on waiting for schema creation * Exclude post-ops from dry run validation query * Handle raw strings in codeEndIndex * Misc fixes * Format * run.ts refactor * Restore and update proto comment * Handle line prefixes for error parsing * Minor formatting issues
Configuration menu - View commit details
-
Copy full SHA for 7b73674 - Browse repository at this point
Copy the full SHA 7b73674View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5ffcfa4 - Browse repository at this point
Copy the full SHA 5ffcfa4View commit details
Commits on Nov 5, 2026
-
Filter stateless-install project copies with ignore-file overrides (#…
…2272) * Filter the stateless-install project copy using .gitignore Fixes #2269. When dataformCoreVersion is set in workflow_settings.yaml, compile() copies the whole project directory to a temporary directory before running `npm i` there. That copy had no filter, so its cost scaled with everything under the project root -- most commonly a large .venv or build-output directory sitting next to definitions/ -- rather than with the Dataform project itself. Adds buildProjectCopyFilter(), an fs-extra copySync filter built from the project's own root .gitignore (already written by `dataform init`, but not previously consulted anywhere in the compile pipeline), plus a small always-ignored floor of .git and node_modules that a .gitignore negation pattern cannot override. Using the project's .gitignore rather than a hardcoded list of directory names avoids having to enumerate every ecosystem's junk directories (.venv, target/, __pycache__/, vendor/, coverage/, ...), since a project's .gitignore already states what that project treats as disposable. Behavior change worth calling out in review: a gitignored file is no longer copied, so it is also no longer compiled. A project that generates definitions into a gitignored path now has to unignore that path. Verbose mode prints which exclusions are applied. Projects with no .gitignore are unaffected beyond the .git/node_modules floor. No lockfile regeneration is needed: `ignore` is already resolved in yarn.lock at 5.3.2 as an existing transitive dependency, and the declared ^5.2.0 range matches that entry. * Address review: .dataformignore support and accurate verbose log - Read an optional .dataformignore from the project root after the .gitignore, into the same ignore instance. It can exclude further paths, or un-ignore gitignored ones with `!pattern` (for example, definitions generated into a gitignored directory). The .git and node_modules floor still cannot be overridden by either file. - The verbose log now names the ignore files actually found, and no longer mentions a .gitignore when the project has none. - Reformat the new files with the repo's Prettier 3 config. * Match ignore patterns case-sensitively; document negation limits - ignore() matches case-insensitively by default, so on a case-sensitive filesystem a `definitions/staging/` pattern also dropped `definitions/Staging/table.sqlx`, which git keeps. Always match case-sensitively: at worst this copies a little extra on case-insensitive filesystems, and never drops a definition. - Document that, as in git, a file can't be re-included while an ancestor directory is still excluded, with a concrete example, and test the child-only negation case. * Harden the copy filter's fixed rules and ignore-file loading - Compare the always-ignored .git/node_modules names case-insensitively, so .GIT or NODE_MODULES on a case-insensitive filesystem is excluded. - Always copy the root workflow_settings.yaml. compile() has already read it from the original project, and compilation can't proceed without it, so a broad pattern like `*.yaml` must not drop it. - Skip a .gitignore/.dataformignore that isn't a regular file, instead of failing the compile with EISDIR. - Document that patterns are evaluated without consulting git's index, so force-added tracked files matching a pattern are still excluded. * Follow the filesystem's case sensitivity in the copy filter A fixed case policy was wrong in both directions: always-sensitive matching let a mixed-case negation miss its file on a case-insensitive filesystem, and always-folding the .git/node_modules floor excluded a legitimate NODE_MODULES directory on Linux. Detect once whether the project directory is case-insensitive (probing an existing entry under its case-swapped name, falling back to the platform default), and apply that to the ignore matcher, the always-ignored floor and the workflow_settings.yaml allowlist, as git does with core.ignorecase. Tests pin each mode explicitly so they behave the same on every platform. * Always copy compilation inputs; drop .dataformignore and case probing Address review on #2272: - Don't apply .gitignore to what compilation reads: definitions/, includes/, workflow_settings.yaml (or dataform.json) and .npmrc, nor to anything reachable from them through symlinks within the project, matching the compile glob, which follows symlinks. Only .git is skipped inside those paths. Everything else is still filtered, so the slow copy from #2269 stays fixed. - Remove .dataformignore support; with compilation inputs always copied, its main use case is gone. - Replace case-sensitivity probing with the platform default: case-insensitive on win32 and darwin, sensitive elsewhere. - Update the verbose log message, and document the copy's behavior in the dataformCoreVersion docs. * Note that caught require() failures can change behavior silently The dataformCoreVersion docs said a skipped file loaded by project code makes compilation fail. Code that catches the error carries on instead, possibly with different behavior, so say that. * Document keeping loaded gitignored files under includes/ A gitignored file that project code loads isn't copied for a stateless-install compile. A subdirectory of includes/ is always copied and isn't loaded as globals, so point users there. * Qualify copy filter claims about git and relocated helpers Nested .gitignore negations can un-ignore files this filter still excludes, so the copy can differ from git in either direction, not just copy more. A caught require() failure changes behavior rather than failing. Moving a helper under includes/ also means moving what it loads by relative path. * Restore .dataformignore and explain modules the copy excluded Some files the stateless-install copy needs aren't compilation inputs: SQL that actions.yaml references outside definitions/, or a CA file .npmrc names by a relative path. When gitignored, they were dropped with no way to keep them short of changing .gitignore or moving them. - Read an optional root .dataformignore after .gitignore, so it can re-include gitignored paths or exclude more. Compilation inputs and the .git/node_modules floor still can't be overridden. - When compilation can't find a module that exists in the project but not in the copy, say which path an ignore file excludes and how to re-include it, instead of the CLI's 'Could not find NPM dependencies' message. - Document both on dataformCoreVersion, and test re-inclusion end to end with an actions.yaml reference. * Simplify the copy filter's error hint and tests - Find the excluded path from the finished copy, which never contains a directory the filter excluded, instead of rebuilding the filter. Check the .git/node_modules floor with the platform's case sensitivity, as the filter does. - Share the stateless-install setup across the end-to-end tests, and drop imports left over from a removed test. - Copy through copyProjectForStatelessInstall in the unit tests, drop a test the others cover, and fold the directory-.gitignore case into the findProjectIgnoreFiles test. - Trim the dataformCoreVersion docs. * Test the graph-error hint and a !* .dataformignore - End to end: a definition that require()s a gitignored helper gets the .dataformignore hint in both --json output and the console's printed stack, and compiles once the helper is re-included. - A .dataformignore of !* re-includes everything the .gitignore excludes, including nested patterns, apart from .git and node_modules. * Make the excluded-module hint descriptive rather than prescriptive The hint suggested one negation to add, which doesn't always recover the file: another pattern can also match it (queries/ and *.sql), and a file name can contain pattern syntax (example[1].sql). It now names the path missing from the copy and points to .dataformignore without proposing a pattern. - Check presence with lstat, so a copied symlink whose target was excluded isn't blamed on the link. - Start from the shallowest path missing from the copy, so an extensionless require("lib/helper") is explained when a directory on its path was excluded. - Say "if the module is there", since an excluded directory doesn't prove the module was in it. - Docs: call the always-copied paths what they are, not compilation inputs, and don't describe .gitignore as listing what's disposable. * Don't explain modules behind copied symlinks; make symlink tests real - Stop explaining a missing module once its path reaches a copied symbolic link. What's missing is behind the link, so naming a path under it (lib/helper.js for lib -> generated) sent users to re-include the wrong thing, and with lib -> .git/lib it blamed ignore files for the fixed .git exclusion. - Under Bazel, rules_nodejs patches fs.readlinkSync to return absolute paths, so links copied in-process pointed back into the original project. Two symlink tests read through those links and passed even with link following disabled. Check each link's target at its own path in the copy instead, and point the hint tests' copied links at their excluded targets in the copy. - Docs: note the .git exception inside definitions/ and includes/. * Print the excluded-module note separately; scan only for symlinks - Leave missing-module errors unchanged and print the note about an excluded path to stderr alongside them, instead of rewriting the error's message and stack. Guard the catch with `instanceof Error`. - Copy anything under the compilation input roots by path prefix. The pre-pass now only lists their directories to find symlinks, recording paths just for what links reach outside them. - Document and test that a directory-only pattern doesn't match a symlink to a directory, as in Git. * Move stateless-install docs to their own page; clean up the temp copy on failure - Shorten the dataformCoreVersion field docs to a summary that links to a new docs/stateless-install.md page. - Remove the temporary copy in a finally block, so it's removed when npm i or compilation fails too, and create it only on the stateless path. - Pass the same case setting to explainExcludedModule as to the copy filter. * Adapt to the node:vm runner's missing-module message The runner names the requiring file after the module, and no longer throws a VMError, so the CLI doesn't report it as a missing npm dependency. * Depend on ignore 7.x 7.0.8 is already in the lockfile, through typescript-eslint. * Don't name the Bazel rules that patch readlinkSync in tests * Correct comments on relative module names and best-effort cleanup * fix(cli): keep temporary environment cleanup best effort A failure to remove the temporary directory mustn't turn a successful compile into a failure, replace the error that ended the compile, or throw from the process exit listener.
Configuration menu - View commit details
-
Copy full SHA for 2576779 - Browse repository at this point
Copy the full SHA 2576779View commit details
Commits on Nov 6, 2026
-
Adapt PropertyGraph graph.yaml schema and DDL compiler to Semantic On…
…tology V1 (#2417) * Adapt PropertyGraph graph.yaml schema and DDL compiler to Semantic Ontology V1 * Assert INFORMATION_SCHEMA.PROPERTY_GRAPHS DDL without Enterprise reservation * Pass property graph field and metric expressions as-is into BigQuery DDL * Reject leading // in property graph source and deployment target URIs * Accept datasets as an alias for entities in property graph config * Validate that property graph field and metric expressions are strings in V1 * Align V1 relationship rules and description folding with Semantic Ontology spec * Address review comments on property graph V1 validation and docs * Expose pages in Book fields for total_pages measure in integration test
Configuration menu - View commit details
-
Copy full SHA for f1a6ef0 - Browse repository at this point
Copy the full SHA f1a6ef0View commit details -
Configuration menu - View commit details
-
Copy full SHA for c846912 - Browse repository at this point
Copy the full SHA c846912View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2d6675b - Browse repository at this point
Copy the full SHA 2d6675bView commit details
Loading
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff 3.1.3...3.1.4