pipeline

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 13, 2026 License: BSD-2-Clause Imports: 34 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BuildEntitiesXML

func BuildEntitiesXML(ids []string, bodies map[string]string, cfg AggregateConfig) []byte

BuildEntitiesXML renders a complete md:EntitiesDescriptor document embedding the full XML body of each entity ID in ids. Missing or empty bodies fall back to a minimal stub EntityDescriptor. cfg attributes are written on the root element; cfg.ValidUntil is resolved via ResolveValidUntil.

func ParseCacheDurationSeconds

func ParseCacheDurationSeconds(cd string) (int, bool)

ParseCacheDurationSeconds converts a subset of ISO 8601 durations (PT[n]H, PT[n]M, PT[n]S, and combinations) to seconds. Returns 0,false for values it cannot parse.

func ResolveValidUntil

func ResolveValidUntil(vu string) string

ResolveValidUntil returns the @validUntil value to write into XML or send as an HTTP Expires header. If vu starts with "+" it is treated as a duration added to the current time; otherwise it is returned unchanged.

Types

type AggregateConfig

type AggregateConfig struct {
	// Name is the @Name attribute; for MDQ endpoints it is typically the
	// aggregate URL (e.g. "https://mdq.example.org/entities").
	Name string

	// CacheDuration is an ISO 8601 duration string (e.g. "PT48H") set as the
	// @cacheDuration attribute and converted to a Cache-Control max-age header.
	CacheDuration string

	// ValidUntil is either an RFC 3339 timestamp or a "+<duration>" offset
	// from the current time (e.g. "+48h").  Written as @validUntil and used
	// to derive an Expires HTTP header.
	ValidUntil string
}

AggregateConfig holds optional SAML metadata attributes applied to the root md:EntitiesDescriptor and the caching parameters derived from them.

type CertReportStep

type CertReportStep struct{}

CertReportStep prints certificate validity information for current entities.

type DiscoEntry

type DiscoEntry struct {
	EntityID             string         `json:"entityID"`
	Type                 []string       `json:"type,omitempty"`
	DisplayNames         []LangValue    `json:"DisplayNames,omitempty"`
	Descriptions         []LangValue    `json:"Descriptions,omitempty"`
	Keywords             []LangKeywords `json:"Keywords,omitempty"`
	Logos                []Logo         `json:"Logos,omitempty"`
	PrivacyStatementURLs []LangValue    `json:"PrivacyStatementURLs,omitempty"`
	InformationURLs      []LangValue    `json:"InformationURLs,omitempty"`
	Domains              []string       `json:"Domains,omitempty"`
	Geo                  []GeoHint      `json:"geo,omitempty"`
}

DiscoEntry is a single entity's representation in the SAML discovery JSON feed.

func BuildDiscoEntries

func BuildDiscoEntries(current []string, attrs map[string]EntityAttributes, xmlDocs map[string]string, roleFilter string) []DiscoEntry

BuildDiscoEntries constructs the discovery JSON feed entries for the given entity set. roleFilter restricts output to entities with the given role ("idp" or "sp"); an empty string returns entries for all entities.

type DiscoJSONStep

type DiscoJSONStep struct {
	Output string `yaml:"output"`
}

DiscoJSONStep writes current entities as a SAML discovery JSON feed.

func (*DiscoJSONStep) UnmarshalYAML

func (d *DiscoJSONStep) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML supports scalar "discojson <path>" and mapping forms.

type EntityAttributes

type EntityAttributes struct {
	Roles                 map[string]struct{}
	Categories            map[string]struct{}
	RegistrationAuthority string
	TextTokens            map[string]struct{}
	IPHints               map[string]struct{}
}

func (*EntityAttributes) AddCategory

func (a *EntityAttributes) AddCategory(cat string)

func (*EntityAttributes) AddIPHint

func (a *EntityAttributes) AddIPHint(v string)

func (*EntityAttributes) AddRole

func (a *EntityAttributes) AddRole(role string)

func (*EntityAttributes) AddTextToken

func (a *EntityAttributes) AddTextToken(v string)

func (EntityAttributes) Clone

func (EntityAttributes) HasCategory

func (a EntityAttributes) HasCategory(cat string) bool

func (EntityAttributes) HasRole

func (a EntityAttributes) HasRole(role string) bool

type ExecuteOptions

type ExecuteOptions struct {
	// Progress, if non-nil, is called after each step with the step index,
	// action name, and a brief status message.
	Progress func(step int, action, msg string)
	// States is the set of active pipeline labels used to evaluate `when`
	// guards.  Nil defaults to {"update": true}, which is the standard batch
	// execution mode.  Set explicitly when invoking the pipeline in a
	// non-update mode (e.g. via-runs use {viaLabel: true}).
	States map[string]bool
	// CryptoExt provides extended algorithm and certificate support
	// (e.g. brainpool curves).
	CryptoExt *cryptoutil.Extensions
}

ExecuteOptions controls optional behavior of Execute.

type File

type File struct {
	Pipeline []Step `yaml:"pipeline"`
	BaseDir  string `yaml:"-"`
}

File represents a pipeline YAML file.

func ParseFile

func ParseFile(path string) (File, error)

ParseFile loads a pipeline YAML file from disk.

type FinalizeStep

type FinalizeStep struct {
	Name          string `yaml:"Name"`
	CacheDuration string `yaml:"cacheDuration"`
	ValidUntil    string `yaml:"validUntil"`
}

FinalizeStep carries metadata applied to XML aggregate output.

type ForkStep

type ForkStep struct {
	Pipeline []Step
}

ForkStep runs a sub-pipeline on a copy of the current state. Changes to the working set inside the fork do not affect the outer pipeline. Primarily used for side-effect outputs (e.g. publishing a sub-aggregate).

type GeoHint

type GeoHint struct {
	Lat float64 `json:"lat"`
	Lon float64 `json:"lon"`
}

GeoHint is a geographic coordinate pair.

type LangKeywords

type LangKeywords struct {
	Lang  string   `json:"lang"`
	Value []string `json:"value"`
}

LangKeywords is a language-tagged list of keyword strings.

type LangValue

type LangValue struct {
	Lang  string `json:"lang"`
	Value string `json:"value"`
}

LangValue is a language-tagged string value.

type LoadStep

type LoadStep struct {
	Files    []string      `yaml:"files"`
	URLs     []string      `yaml:"urls"`
	Sources  []SourceEntry `yaml:"sources"`
	Verify   string        `yaml:"verify"`
	Timeout  string        `yaml:"timeout"`
	Retries  int           `yaml:"retries"`
	Cleanup  bool          `yaml:"cleanup"`
	Entities []string      `yaml:"entities"`
	Via      []string      `yaml:"via"`
	From     string        `yaml:"from"`
	// AllowPrivateAddrs disables the private-IP SSRF blocklist for all URL
	// sources within this load step. Use for intentionally internal endpoints.
	AllowPrivateAddrs bool `yaml:"allow_private_addrs"`
}

LoadStep loads metadata into the pipeline. Resources are given directly as files, URLs, or inline entity IDs. In-pipeline aliases produced by "select as /name" can be referenced in Files. Source entries in Sources support per-source aliases and cert verification. From registers the loaded entity set under the given alias name in the source map after loading (pyFF compat: makes the loaded data addressable by name).

func (*LoadStep) UnmarshalYAML

func (l *LoadStep) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML supports: - scalar: URL or file path to load (e.g. "- load http://example.org/fed.xml") - sequence: list of URLs/file paths - mapping: {files: [...], urls: [...], entities: [...], ...}

type Logo struct {
	Value  string `json:"value"`
	Width  int    `json:"width,omitempty"`
	Height int    `json:"height,omitempty"`
	Lang   string `json:"lang,omitempty"`
}

Logo is an MDUI logo entry with URL, dimensions, and optional language tag.

type NodeCountryStep

type NodeCountryStep struct{}

NodeCountryStep enriches entities with country tokens from embedded X.509 certs.

type PKCS11SignSettings

type PKCS11SignSettings struct {
	ModulePath string `yaml:"module_path"`
	SlotID     uint   `yaml:"slot_id"`
	// PIN is the literal slot user PIN. Prefer PINEnv or PINFile to avoid
	// storing secrets in the pipeline YAML file.
	PIN string `yaml:"pin"`
	// PINEnv names an environment variable whose value is the slot PIN.
	PINEnv string `yaml:"pin_env"`
	// PINFile is the path to a file whose first line contains the slot PIN.
	PINFile  string `yaml:"pin_file"`
	KeyLabel string `yaml:"key_label"`
	KeyID    string `yaml:"key_id"`
}

PKCS11SignSettings mirrors PKCS#11 signer configuration.

type PubInfoStep

type PubInfoStep struct {
	Publisher string   `yaml:"publisher"`
	Value     string   `yaml:"value"`
	Values    []string `yaml:"values"`
	URL       string   `yaml:"url"`
	URLs      []string `yaml:"urls"`
	Lang      string   `yaml:"lang"`
}

PubInfoStep applies publication-related text metadata to current entities.

type PublishStep

type PublishStep struct {
	Output      string `yaml:"output"`
	As          string `yaml:"as"`
	Resource    string `yaml:"resource"`
	Dir         string `yaml:"dir"`
	HashLink    bool   `yaml:"hash_link"`
	UpdateStore bool   `yaml:"update_store"`
	StoreDir    string `yaml:"store_dir"`
	URLEncode   bool   `yaml:"urlencode_filenames"`
	Ext         string `yaml:"ext"`
	Raw         bool   `yaml:"raw"`
}

PublishStep writes current entities to an output file. URLEncode, Ext, and Raw are supported for directory publishing:

  • URLEncode: write MDQ-compatible URL-encoded {sha256}HEX filenames
  • Ext: file extension for directory items (default ".xml")
  • Raw: accepted for pyFF compatibility; dir publish is always raw

func (*PublishStep) UnmarshalYAML

func (p *PublishStep) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML supports: - scalar: output file path - mapping: structured publish arguments

type RegInfoStep

type RegInfoStep struct {
	Authority             string   `yaml:"authority"`
	RegistrationAuthority string   `yaml:"registration_authority"`
	Policy                string   `yaml:"policy"`
	Policies              []string `yaml:"policies"`
	Selector              string   `yaml:"selector"`
}

RegInfoStep applies registration authority metadata to current entities. If Selector is set, enrichment is applied only to entities matching the selector.

type Result

type Result struct {
	Entities  []string
	EntityXML map[string]string
	Attrs     map[string]EntityAttributes
	Finalize  FinalizeStep
	Sign      SignStep
	Verify    VerifyStep
	// DiscoJSON holds the entries built by the most recent discojson/discojson_idp/
	// discojson_sp step.  Nil when no discojson step was executed.  The MDQ server
	// uses this to serve the discovery feed in-memory without reading the on-disk
	// file written by the step.
	DiscoJSON []DiscoEntry
}

Result contains final in-memory entities after execution.

func Execute

func Execute(p File, outputDir string, opts ...ExecuteOptions) (Result, error)

Execute runs a parsed pipeline file.

type SelectStep

type SelectStep struct {
	Entities              []string `yaml:"entities"`
	Selector              string   `yaml:"selector"`
	Selectors             []string `yaml:"selectors"`
	As                    string   `yaml:"as"`
	Dedup                 *bool    `yaml:"dedup"`
	Role                  string   `yaml:"role"`
	Roles                 []string `yaml:"roles"`
	EntityCategory        string   `yaml:"entity_category"`
	EntityCategories      []string `yaml:"entity_categories"`
	RegistrationAuthority string   `yaml:"registration_authority"`
	Match                 string   `yaml:"match"`
}

SelectStep filters current entities to the provided set.

func (*SelectStep) UnmarshalYAML

func (s *SelectStep) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML supports: - scalar: selector expression or entity ID - sequence: list of selector expressions - mapping: structured select arguments

type SetAttrStep

type SetAttrStep struct {
	Name     string   `yaml:"name"`
	Value    string   `yaml:"value"`
	Values   []string `yaml:"values"`
	Selector string   `yaml:"selector"`
}

SetAttrStep applies metadata-like attribute enrichments to current entities. If Selector is set, enrichment is applied only to entities matching the selector.

type SignStep

type SignStep struct {
	Key    string              `yaml:"key"`
	Cert   string              `yaml:"cert"`
	PKCS11 *PKCS11SignSettings `yaml:"pkcs11"`
}

SignStep configures XML signing for published XML output.

type SortStep

type SortStep struct {
	OrderBy string `yaml:"order_by"`
}

SortStep configures entity sorting behavior.

func (*SortStep) UnmarshalYAML

func (s *SortStep) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML supports: - scalar: xpath used as order_by - mapping: structured sort arguments

type Source

type Source struct {
	ID       string   `yaml:"id"`
	Entities []string `yaml:"entities"`
	Files    []string `yaml:"files"`
	URLs     []string `yaml:"urls"`
	Verify   string   `yaml:"verify"`
	Timeout  string   `yaml:"timeout"`
	Retries  int      `yaml:"retries"`
	Cleanup  bool     `yaml:"cleanup"`
	// MaxBytes caps the size of each HTTP response body. Defaults to 100 MiB (104857600) when zero.
	MaxBytes int64 `yaml:"max_bytes"`
	// AllowPrivateAddrs disables the private-IP / cloud-metadata SSRF blocklist for this source.
	// Enable only for intentionally internal or intranet federation endpoints.
	AllowPrivateAddrs bool `yaml:"allow_private_addrs"`
}

Source is a named metadata source used by load steps.

type SourceEntry

type SourceEntry struct {
	URL     string `yaml:"url"`
	File    string `yaml:"file"`
	As      string `yaml:"as"`
	Via     string `yaml:"via"`
	Verify  string `yaml:"verify"`
	Cleanup bool   `yaml:"cleanup"`
}

SourceEntry is a single source item within a LoadStep, supporting per-source aliases (as), preprocessing branches (via), cert verification (verify), and cleanup flags.

type StatsStep

type StatsStep struct{}

StatsStep configures stats output behavior. Empty configuration uses defaults.

type Step

type Step struct {
	Action      string
	Load        LoadStep
	Select      SelectStep
	Filter      SelectStep
	Pick        SelectStep
	SetAttr     SetAttrStep
	RegInfo     RegInfoStep
	PubInfo     PubInfoStep
	Sort        SortStep
	Finalize    FinalizeStep
	Sign        SignStep
	Verify      VerifyStep
	Publish     PublishStep
	Store       StoreStep
	Stats       StatsStep
	NodeCountry NodeCountryStep
	CertReport  CertReportStep
	DiscoJSON   DiscoJSONStep
	XSLT        XSLTStep
	Fork        ForkStep
	When        WhenStep
	Then        string // label for `then <label>:` steps — re-runs root pipeline with {label:true}
}

Step is one pipeline operation.

func (*Step) UnmarshalYAML

func (s *Step) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML supports two formats: - scalar: "load" - mapping: {load: {source: federation}}

type StoreStep

type StoreStep struct {
	Directory string `yaml:"directory"`
}

StoreStep writes the current entity set to a content-addressable on-disk directory, equivalent to pyFF's `store: directory:` pipe. goFF executes it as a `publish: {dir: Directory}` step.

func (*StoreStep) UnmarshalYAML

func (s *StoreStep) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML supports three pyFF forms:

store: directory: /path          (mapping)
store: - directory /path         (sequence scalar)
store: /path                     (bare scalar)

type VerifyStep

type VerifyStep struct {
	Cert  string   `yaml:"cert"`
	Certs []string `yaml:"certs"`
	// CheckExpiry enables NotBefore/NotAfter validation on the verification certificates.
	// Disabled by default to support the common SAML practice of pinning known certs
	// independent of their PKI validity period.
	CheckExpiry bool `yaml:"check_expiry"`
}

VerifyStep configures XML signature verification for published XML output. Either Cert (single PEM file path) or Certs (list of PEM file paths) may be set; both are accepted and merged into the verification certificate pool. Multiple certificates allow key-rollover pipelines where the signer may use either of two valid certs.

type WhenStep

type WhenStep struct {
	Condition string   // e.g. "update", "normalize", "accept"
	Values    []string // optional multi-word values, e.g. ["application/json"]
	Body      []Step
}

WhenStep represents a conditional guard: execute Body only when Condition is present (with optional Values) in the pipeline's active state labels. This models pyFF's `when <condition> [values]:` pipe.

type XSLTStep

type XSLTStep struct {
	Stylesheet string `yaml:"stylesheet"`
}

XSLTStep applies an XSLT transformation to the current entity set using xsltproc.

func (*XSLTStep) UnmarshalYAML

func (x *XSLTStep) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML supports: - scalar: stylesheet file path ("xslt transform.xsl") - mapping: structured arguments ({stylesheet: transform.xsl})

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址