stefunny

stefunny

stefunny is a deployment tool for AWS StepFunctions state machine and the accompanying AWS EventBridge rule and scheudle.

stefunny does,

• Create a state machine.
• Create a EventBridge rule and EventBridge Scheduler schedule.
• Deploy state machine/ EventBridge rule / EventBridge Scheduler schedule/ StateMachine Alias.
• Rollback to the previous version of the state machine.
• Manage state machine versions.
• Show status of the state machine.

That's all for now.

stefunny does not,

• Manage resources related to the StepFunctions state machine.
  • e.g. IAM Role, Resources called by state machine, Trigger rule that is not a schedule, CloudWatch LogGroup, etc...
• Manage StepFunctions Activities and Activity Worker.

If you hope to manage these resources collectively, we recommend other deployment tools (AWS SAM, Serverless Framework, etc.).

If you hope to manage these resources partially individually, we recommend the following tools:

• terraform for IAM Role, CloudWatch LogGroups, etc...
• lambroll for AWS Lambda function.
• ecspresso for AWS ECS Task.

Install

Homebrew (macOS and Linux)

$ brew install mashiike/tap/stefunny

aqua

aqua is a declarative CLI Version Manager.

$ aqua g -i mashiike/stefunny

Binary packages

Releases

GitHub Actions

Action mashiike/stefunny@v0 installs stefunny binary for Linux into /usr/local/bin. This action runs install only.

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: mashiike/stefunny@v0
        with:
          version: v0.6.0 
      - run: |
          stefunny deploy

QuickStart

Try migrate your existing StepFunctions StateMachine to stefunny.

$ mkdir hello
$ cd hello
$ stefunny init --state-machine Hello     
2024/02/13 16:07:53 [notice] StateMachine/Hello save config to /home/user/hello/stefunny.yaml
2024/02/13 16:07:53 [notice] StateMachine/Hello save state machine definition to /home/user/hello/definition.asl.json

Edit the definition.asl.json and stefunny.yaml.

Now you can deploy state machine Hello using stefunny deploy.

$ stefunny deploy
2024/02/13 16:18:42 [info] Starting deploy 
2024/02/13 16:18:42 [info] update state machine `arn:aws:states:ap-northeast-1:123456789012:stateMachine:Hello:2`
2024/02/13 16:18:42 [info] update current alias `arn:aws:states:ap-northeast-1:123456789012:stateMachine:Hello:current`
2024/02/13 16:18:42 [info] deploy state machine `Hello`(at `2024-02-13 07:17:48.178 +0000 UTC`)
2024/02/13 16:18:43 [info] finish deploy 

Usage

Usage: stefunny <command>

stefunny is a deployment tool for AWS StepFunctions state machine

Flags:
  -h, --help                      Show context-sensitive help.
      --log-level="info"          Set log level (debug, info, notice, warn, error) ($STEFUNNY_LOG_LEVEL)
  -c, --config="stefunny.yaml"    Path to config file ($STEFUNNY_CONFIG)
      --tfstate=STRING            URL to terraform.tfstate referenced in config ($STEFUNNY_TFSTATE)
      --ext-str=,...              external string values for Jsonnet
      --ext-code=,...             external code values for Jsonnet
      --region=""                 AWS region ($AWS_REGION)
      --alias="current"           Alias name for state machine ($STEFUNNY_ALIAS)

Commands:
  version
    Show version

  init --state-machine=STRING
    Initialize stefunny configuration

  delete
    Delete state machine and schedule rules

  deploy
    Deploy state machine and schedule rules

  rollback
    Rollback state machine

  schedule --enabled --disabled
    Enable or disable schedule rules (deprecated)

  render <targets> ...
    Render state machine definition

  execute
    Execute state machine

  versions
    Manage state machine versions

  diff
    Show diff of state machine definition and trigers

  pull
    Pull state machine definition

  studio
    Show Step Functions workflow studio URL

  status
    Show status of state machine

Run "stefunny <command> --help" for more information on a command.

Init

stepfunny init initialize stefunny.yaml and definition file by existing state machine.

Usage: stefunny init --state-machine=STRING

Initialize stefunny configuration

Flags:
  -h, --help                                Show context-sensitive help.
      --log-level="info"                    Set log level (debug, info, notice, warn, error) ($STEFUNNY_LOG_LEVEL)
  -c, --config="stefunny.yaml"              Path to config file ($STEFUNNY_CONFIG)
      --tfstate=STRING                      URL to terraform.tfstate referenced in config ($STEFUNNY_TFSTATE)
      --ext-str=,...                        external string values for Jsonnet
      --ext-code=,...                       external code values for Jsonnet
      --region=""                           AWS region ($AWS_REGION)
      --alias="current"                     Alias name for state machine ($STEFUNNY_ALIAS)

      --state-machine=STRING                AWS StepFunctions state machine name ($STATE_MACHINE_NAME)
  -d, --definition="definition.asl.json"    Path to state machine definition file ($DEFINITION_FILE_PATH)
      --env=ENV,...                         templateize environment variables
      --must-env=MUST-ENV,...               templateize must environment variables
      --skip-trigger                        Skip trigger

created file foramt are checked file extension. .json saved as json, .jsonnet saved as jsonnet, .yaml or .yml saved as yaml.

If you manage the aws resources by terraform, you can use --tfstate flag with stefunny init command.

$ export ENV=dev
$ stefunny init --state-machine dev-Hello --tfstate s3://my-bucket/terraform.tfstate --must-env ENV

in this case, saved config and definition file are templatized by text/template

for example, the saved config file is like this.

aws_region: ap-northeast-1
required_version: ">=v0.6.0"
state_machine:
  definition: definition.asl.json
  logging_configuration:
    destinations:
    - cloudwatch_logs_log_group:
        log_group_arn: "{{ tfstate `aws_cloudwatch_log_group.state_machine.arn` }}:*"
    include_execution_data: true
    level: ALL
  name: "{{ must_env `ENV` }}-Hello"
  role_arn: "{{ tfstate `aws_iam_role.state_machine.arn` }}"
  tags:
  - key: Name
    value: "{{ must_env `ENV` }}-Hello"
  tracing_configuration:
    enabled: true
  type: STANDARD

Deploy

Usage: stefunny deploy

Deploy state machine and schedule rules

Flags:
  -h, --help                          Show context-sensitive help.
      --log-level="info"              Set log level (debug, info, notice, warn, error) ($STEFUNNY_LOG_LEVEL)
  -c, --config="stefunny.yaml"        Path to config file ($STEFUNNY_CONFIG)
      --tfstate=STRING                URL to terraform.tfstate referenced in config ($STEFUNNY_TFSTATE)
      --ext-str=,...                  external string values for Jsonnet
      --ext-code=,...                 external code values for Jsonnet
      --region=""                     AWS region ($AWS_REGION)
      --alias="current"               Alias name for state machine ($STEFUNNY_ALIAS)

      --dry-run                       Dry run
      --skip-state-machine            Skip deploy state machine
      --skip-trigger                  Skip deploy trigger
      --version-description=STRING    Version description
      --keep-versions=0               Number of latest versions to keep. Older versions will be deleted. (Optional value: default 0)
      --trigger-enabled               Enable trigger
      --trigger-disabled              Disable trigger
      --[no-]unified                  when dry run, output unified diff

stefunny deploy works as below.

• Create / Update State Machine from config file and definition file(yaml/json/jsonnet)
  • Replace {{ env FOO bar }} syntax in the config file and definition file to environment variable "FOO". If "FOO" is not defined, replaced by "bar"
  • Replace {{ must_env FOO }} syntax in the config file and definition file to environment variable "FOO". If "FOO" is not defined, abort immediately.
  • If a terraform state is given in config, replace the {{tfstate <tf resource name>}} syntax in the config file and definition file with reference to the state content.
• Publish new version of the state machine.
• Update the alias to the new version.
• Create/ Update EventBridge rule.
• Create/ Update EventBridge Scheduler schedule.

stefunny deploy always applies the current config as-is; it has no notion of "there is no diff, so skip". It always publishes a new state machine version and moves the alias, even when nothing changed. If you want to skip deploy when there is no change, check for a diff before calling deploy, e.g. with stefunny diff --exit-code in CI (see Diff below).

Diff

Usage: stefunny diff [flags]

Show diff of state machine definition and trigers

Flags:
  -h, --help                      Show context-sensitive help.
      --log-level="info"          Set log level (debug, info, notice, warn, error) ($STEFUNNY_LOG_LEVEL)
  -c, --config="stefunny.yaml"    Path to config file ($STEFUNNY_CONFIG)
      --tfstate=STRING            URL to terraform.tfstate referenced in config ($STEFUNNY_TFSTATE)
      --ext-str=,...              external string values for Jsonnet
      --ext-code=,...             external code values for Jsonnet
      --region=""                 AWS region ($AWS_REGION)
      --alias="current"           Alias name for state machine ($STEFUNNY_ALIAS)

  -u, --[no-]unified              output in unified format
      --qualifier=""              qualifier for state machine
      --exit-code                 exit with code 2 if there are differences
      --skip-trigger              Skip trigger diff

stefunny diff shows the diff of the state machine definition and triggers (EventBridge rules / EventBridge Scheduler schedules) between the config and the deployed resources.

--skip-trigger skips the trigger diff (EventBridge rules / EventBridge Scheduler schedules), mirroring deploy --skip-trigger. Use it when triggers are managed outside of stefunny (e.g. by Terraform), so diff --exit-code reflects only what deploy --skip-trigger would actually change.

With --exit-code, the exit status reports whether a diff was found, matching lambroll's diff --exit-code: exit code 0 when there is no diff, 2 when there is a diff, and any other non-zero code on an actual error. This lets CI gate deploy on whether anything changed, without deploy itself having to decide:

- name: diff
  id: diff
  run: |
    set +e
    stefunny diff --exit-code
    code=$?
    set -e
    case $code in
      0) echo "changed=false" >> "$GITHUB_OUTPUT" ;;
      2) echo "changed=true"  >> "$GITHUB_OUTPUT" ;;
      *) exit $code ;;
    esac

- name: deploy
  if: steps.diff.outputs.changed == 'true'
  run: stefunny deploy --keep-versions 10

Rollback

Usage: stefunny rollback

Rollback state machine

Flags:
  -h, --help                      Show context-sensitive help.
      --log-level="info"          Set log level (debug, info, notice, warn, error) ($STEFUNNY_LOG_LEVEL)
  -c, --config="stefunny.yaml"    Path to config file ($STEFUNNY_CONFIG)
      --tfstate=STRING            URL to terraform.tfstate referenced in config ($STEFUNNY_TFSTATE)
      --ext-str=,...              external string values for Jsonnet
      --ext-code=,...             external code values for Jsonnet
      --region=""                 AWS region ($AWS_REGION)
      --alias="current"           Alias name for state machine ($STEFUNNY_ALIAS)

      --dry-run                   Dry run
      --keep-version              Keep current version, no delete

stefunny deploy create/update alias current to the published state machine version on deploy.

stefunny rollback works as below.

1. Find previous one version of state machine.
2. Update alias current to the previous version.
3. default delete old version of state machine. (when --keep-version specified, not delete old version of state machine)

Studio and Pull

If you use AWS Step Functions Workflow Studio, you can open the studio URL with stefunny studio command.

Usage: stefunny studio

Show Step Functions workflow studio URL

Flags:
  -h, --help                      Show context-sensitive help.
      --log-level="info"          Set log level (debug, info, notice, warn, error) ($STEFUNNY_LOG_LEVEL)
  -c, --config="stefunny.yaml"    Path to config file ($STEFUNNY_CONFIG)
      --tfstate=STRING            URL to terraform.tfstate referenced in config ($STEFUNNY_TFSTATE)
      --ext-str=,...              external string values for Jsonnet
      --ext-code=,...             external code values for Jsonnet
      --region=""                 AWS region ($AWS_REGION)
      --alias="current"           Alias name for state machine ($STEFUNNY_ALIAS)

      --open                      open workflow studio

stefunny studio command shows the studio URL. If --open flag is specified, open the studio URL in the browser. Edit state machine on Workflow Studio, and pull the definition file with stefunny pull command.

Usage: stefunny pull

Pull state machine definition

Flags:
  -h, --help                      Show context-sensitive help.
      --log-level="info"          Set log level (debug, info, notice, warn, error) ($STEFUNNY_LOG_LEVEL)
  -c, --config="stefunny.yaml"    Path to config file ($STEFUNNY_CONFIG)
      --tfstate=STRING            URL to terraform.tfstate referenced in config ($STEFUNNY_TFSTATE)
      --ext-str=,...              external string values for Jsonnet
      --ext-code=,...             external code values for Jsonnet
      --region=""                 AWS region ($AWS_REGION)
      --alias="current"           Alias name for state machine ($STEFUNNY_ALIAS)

      --[no-]templateize          templateize output
      --qualifier=STRING          qualifier for the version

stefunny pull command pull the definition file from the state machine and save it to the file.

config file (yaml)

required_version: ">=v0.6.0"
aws_region: "{{ env `AWS_REGION` `ap-northeast-1` }}"

state_machine:
  name: "{{ must_env `ENV` }}-Hello"
  definition: definition.asl.json
  logging_configuration:
    destinations:
      - cloudwatch_logs_log_group:
          log_group_arn: "{{ tfstate `aws_cloudwatch_log_group.state_machine.arn` }}:*"
    include_execution_data: true
    level: ALL

  role_arn: "{{ tfstate `aws_iam_role.state_machine.arn` }}"
  tracing_configuration:
    enabled: true
  type: STANDARD

tfstate:
  - location: s3://my-tfstate-bucket/terraform.tfstate

trigger:
  schedule:
    - name: "{{ must_env `ENV` }}-stefunny-test"
      group_name: default
      action_after_completion: DELETE
      flexible_time_window:
        maximum_window_in_minutes: 240.0
        mode: FLEXIBLE
      schedule_expression: at(2024-02-29T00:01:00)
      target:
        retry_policy:
          maximum_event_age_in_seconds: 86400.0
          maximum_retry_attempts: 185.0
        role_arn: "{{ tfstate `aws_iam_role.event_bridge_scheduler.arn` }}"

  event:
    - name: "{{ must_env `ENV` }}-stefunny-test"
      event_bus_name: default
      event_pattern: "{{ file `event_pattern.json` | json_escape }}"
      role_arn: "{{ tfstate `aws_iam_role.event_bridge.arn` }}"

Configuration files and definition files are read with text/template, stefunny has template functions env, must_env, file, json_escape, tfstate and caller_identity.

Template syntax

stefunny uses the text/template standard package in Go to render template files, and parses as YAML/JSON/Jsonnet.

env

"{{ env `NAME` `default value` }}"

If the environment variable NAME is set, it will replace with its value. If it's not set, it will replace with "default value".

must_env

"{{ must_env `NAME` }}"

It replaces with the value of the environment variable NAME. If the variable isn't set at the time of execution, stefunny will panic and stop forcefully.

By defining values that can cause issues when running without meaningful values with must_env, you can prevent unintended deployments.

json_escape

"{{ must_env `JSON_VALUE` | json_escape }}"

It escapes values as JSON strings. Use it when you want to escape values that need to be embedded as strings and require escaping, like quotes.

file

"{{ file `path/to/file` }}"

tfstate

If written tfstate section in the configuration file, it will be use tfstate template function. as following.

stefunny.yaml

required_version: ">v0.0.0"

state_machine:
  name: send_sns
  definition: send_sns.asl.jsonnet
  role_arn: "{{ tfstae `aws_iam_role.stepfunctions.arn` }}"
  logging_configuration:
    level: OFF

tfstate:
  - path: "./terraform.tfstate"
  - url: s3://my-bucket/terraform.tfstate
    func_prefix: s3_

send_sns.asl.jsonnet

{
  Comment: "A simple AWS Step Functions state machine that sends a message to an SNS topic", 
  StartAt: "Send SNS Message",
  States: {
    "Send SNS Message": {
      Type: "Task",
      Resource: "arn:aws:states:::sns:publish",
      Parameters: {
        "TopicArn": "{{ s3_tfstate `aws_sns_topic.topic.arn` }}",
        "Message.$": "$"
      },
      End: true,
    }
  }
}

{{ tfstate "resource_type.resource_name.attr" }} will expand to an attribute value of the resource in tfstate.

{{ tfstatef "resource_type.resource_name['%s'].attr" "index" }} is similar to {{ tfstatef "resource_type.resource_name['index'].attr" }}. This function is useful to build a resource address with environment variables.

{{ tfstatef `aws_subnet.ecs['%s'].id` (must_env `SERVICE`) }}

This function uses tfstate-lookup to load tfstate.

caller_identity

caller_identity resolves the AWS account calling stefunny (the same information as aws sts get-caller-identity). It is available both as a text/template function and, in Jsonnet definition/config files, as a native function via std.native.

"{{ caller_identity.Account }}"

std.native('caller_identity')().Account

It returns an object with Account, Arn and UserId fields. The result is resolved at most once per stefunny invocation and reused for every subsequent reference.

Using caller_identity makes config/definition loading require resolvable AWS credentials, even for commands like render that would otherwise not need any AWS access. If a config doesn't use caller_identity (and doesn't rely on the deprecated state_machine.logging field with a non-ARN log group, which resolves AWS credentials for an unrelated reason), loading it never touches AWS.

If caller_identity is used inside the configuration file itself (not only in the definition file), it is resolved before that file has finished decoding, so aws_region and endpoints.sts written in the same file are not yet available; set AWS_REGION / AWS_ENDPOINT_URL_STS via environment variables instead if you need to control them in that case.

Because the result is memoized for the whole invocation, this also means: if caller_identity is used in the configuration file, the definition file reuses that same already-resolved value rather than re-resolving it with the now-decoded aws_region / endpoints.sts. In practice this rarely matters, since Account, Arn and UserId don't depend on region or endpoint.

Special Thanks

@fujiwara has given me naming idea of stefunny.

Inspire tools

• lambroll
• ecspresso

LICENSE

MIT License

Copyright (c) 2021 IKEDA Masashi