Skip to content

fix: recover from panic when Jellyfin is unreachable during auth - #62

Merged
jay739 merged 1 commit into
mainfrom
fix/jellyfin-auth-panic-on-unreachable
Jul 16, 2026
Merged

jay739 merged 1 commit into
mainfrom
fix/jellyfin-auth-panic-on-unreachable

Conversation

@jay739

@jay739 jay739 commented Jul 16, 2026

Copy link
Copy Markdown
Owner

Summary

  • hrfee/mediabrowser's Authenticate() does an unchecked type assertion on the response body (respData["AccessToken"].(string)) and panics with a nil pointer dereference when Jellyfin is unreachable and returns a non-JSON body instead of a clean connection error
  • Gin's global recovery middleware catches this so the container doesn't crash, but it only surfaces as a generic 500 ("errorInternalServer"), not a clear "Jellyfin is unreachable" message
  • With jellyfin_login = true, every admin login goes through this path — a Jellyfin outage currently blocks admin access to the panel entirely, including for sending downtime notices to users, which defeats the point of having an admin panel that's meant to be usable independently of Jellyfin's own uptime
  • Found and fixed a second, identical unguarded call site: ChangeMyPassword in api-userpage.go (user-facing "change my Jellyfin password" flow) has the same panic risk

Changes

  • auth.go: validateJellyfinCredentials now recovers from the panic and returns a clean 503 ("Jellyfin unreachable") instead
  • api-userpage.go: ChangeMyPassword wraps its Authenticate call in the same recover-and-respond pattern (503 via respondBool), scoped to just that call so it doesn't swallow panics from the rest of the handler

Related

Relates to #56 (audit for remaining upstream jfa-go traces / robustness gaps) — same category of "this fork should hold up under real self-hosted failure conditions" issue.

Test plan

  • Stop the Jellyfin container, confirm admin login returns a 503 with a clear message instead of a 500
  • Confirm normal login still works with Jellyfin up (no regression to the happy path)
  • Confirm ChangeMyPassword behaves the same way with Jellyfin down
  • CI build/test passes (no local Go toolchain available to compile-check ahead of push)

hrfee/mediabrowser's Authenticate() does an unchecked type assertion on
the response body (respData["AccessToken"].(string)) and panics with a
nil pointer dereference when Jellyfin is unreachable and returns a
non-JSON body instead of a clean connection error. Gin's global recovery
middleware was catching this, but only as a generic 500, and with
jellyfin_login enabled every admin login goes through this path, so a
Jellyfin outage blocks admin access to the panel entirely, including for
sending downtime notices to users, which defeats the point of the admin
panel being separate from Jellyfin in the first place.

Adds a local recover() around both call sites (validateJellyfinCredentials
in auth.go, and the password-confirmation call in ChangeMyPassword) that
turns the panic into a clean 503 instead.
@jay739
jay739 merged commit d4a158b into main Jul 16, 2026
7 checks passed
@jay739
jay739 deleted the fix/jellyfin-auth-panic-on-unreachable branch July 16, 2026 04:08
jay739 added a commit that referenced this pull request Jul 16, 2026
…63)

app.jf.MustAuthenticate previously called Fatalf on failure, so if
Jellyfin was down or unreachable at container boot, omnifin would
retry a few times then die -- and with a restart-unless-stopped policy,
crash-loop indefinitely until Jellyfin came back. This defeats the
point of the admin panel being usable independently of Jellyfin's own
uptime (e.g. sending downtime announcements to users while Jellyfin is
mid-maintenance).

Checked the vendored hrfee/mediabrowser client source directly: the
unchecked type assertion that causes a panic (fixed at both call sites
in #62) exists in exactly one place, Authenticate(). Every other
MediaBrowser method returns an ordinary Go error, so failing to
authenticate at startup just means later app.jf.* calls will keep
returning clean errors (some still surfacing as generic 500s -- a
separate, lower-urgency error-message cleanup, not a stability risk)
until Jellyfin becomes reachable, instead of the whole app never
starting.
jay739 added a commit that referenced this pull request Jul 16, 2026
…64)

Found live 2026-07-16 while testing #63: the housekeeping and user
daemons were panicking every ~60s cycle. hrfee/mediabrowser's
GetUsers/UserByID/UserByName internally call Authenticate() themselves
whenever the client isn't currently authenticated -- the same unchecked
type assertion that panics on an unreachable Jellyfin (fixed at its two
explicit call sites in #62) is therefore reachable from any app.jf.*
call, not just login/password-change, whenever Jellyfin has been
unreachable since the last successful auth. That's now the normal state
after #63 made a failed startup auth non-fatal, so every daemon cycle
and every handler that reads Jellyfin user data was panicking silently,
recovered only by gin's global middleware.

Adds safeGetUsers/safeUserByID/safeUserByName wrappers on appContext
that recover locally and return a clean error, and switches every call
site across the app (15 files) from the raw app.jf.* method to the
wrapped version. The wrappers themselves still call the real app.jf.*
methods internally, so behavior when Jellyfin IS reachable is unchanged.
jay739 added a commit that referenced this pull request Jul 16, 2026
Found live 2026-07-16 after #65 shipped and was verified deployed:
panic bursts continued unchanged. Re-audited the ENTIRE
hrfee/mediabrowser library source this time (every .go file, not just
the ones already touched) for the "if !mb.Authenticated { Authenticate()
}" lazy re-auth pattern that panics on an unreachable Jellyfin --
previous passes only checked users_shared.go and missed 3 more
occurrences: GetActivityLog (system_shared.go), DeleteUser and NewUser
(mediabrowser.go). That's 5 total unsafe entry points in the whole
library; #62/#64/#65 covered 2 of them.

GetActivityLog is the actual source of tonight's recurring panic
bursts: JFActivityCache.MaybeSync calls it on every cache refresh,
triggered whenever the admin panel's activity widget polls (every 30s
per admin.ts) or any handler reads activity data.

Adds safeGetActivityLog (jf_activity.go, since activityLogEntrySource
is a locally-scoped interface, not appContext) and
safeNewUser/safeDeleteUser (auth.go, alongside the existing wrappers)
following the same pattern. Grepped the whole codebase once more for
every one of the 5 unsafe methods -- confirmed every remaining raw call
site is either a wrapper's own body or genuinely unreachable from a
live request (the interactive `omnifin test` CLI tool, dead/commented
code).
jay739 added a commit that referenced this pull request Jul 16, 2026
Root cause of tonight's whole investigation, finally traced with a real
stack trace instead of log-timing inference: the panic in Authenticate()
(unchecked type assertion, #62) is recovered INSIDE hrfee/mediabrowser
itself, by a defer mb.timeoutHandler() registered before the panic-prone
line, wired up via mediabrowser.NewNamedTimeoutHandler(..., noFail=true)
in main.go. It logs "Failed to authenticate with Jellyfin @ ..." via the
stdlib log package directly (not app.err/app.debug, which is why this
was so hard to trace across #64/#65/#66 -- none of my recover() wrappers
in those PRs ever had anything to catch, because the panic never
reaches them).

Authenticate() returns afterward with a NIL error and a ZERO-VALUE User{},
since the panic happens before mb.Authenticated = true and before the
real return statement execute. Every err != nil check downstream sees a
clean success and proceeds with garbage/empty user data -- this is why
/token/login returned a plain 500 (CreateToken failing on an empty
userID) with zero panic trace anywhere, and why #64/#65/#66 never
changed the observed behavior at all.

Fixes validateJellyfinCredentials (the actual login path) to check
app.authJf.Authenticated after the call, not just err -- that's the one
reliable signal the library sets only on genuine success. Kept the
existing defer/recover() as a backstop in case a future library version
removes its own internal recovery.

Scope note: #64/#65/#66's wrapper functions (safeGetUsers/safeUserByID/
etc.) likely still have latent versions of this same gap wherever they
wrap a method that internally lazy-reauthenticates, but GetUsers/
UserByID/UserByName/NewUser/DeleteUser proceed to their own real HTTP
call after a swallowed auth failure, which should independently surface
a real error from THAT call instead of silently succeeding -- unlike
Authenticate() itself, which has nothing left to do after auth. Left
as a follow-up to verify/harden rather than expanding this fix further
tonight.
jay739 added a commit that referenced this pull request Jul 16, 2026
Found live 2026-07-16, the second time in one night: the admin panel's
Users tab is completely unusable during a Jellyfin outage, even after
fixing the underlying panics/500s (#62-#71) -- GetUsers/SearchUsers
fundamentally need a live Jellyfin connection to list users, since
Jellyfin IS the user database. There's no way to make that endpoint
work without Jellyfin reachable.

What CAN work: app.storage's own persistent badger DB already stores
an EmailAddress record (Jellyfin ID, email, label, contact/admin
flags) per user, independent of Jellyfin's own availability. This was
previously only accessible via internal code paths (housekeeping
cleanup, migrations) with no API route exposing it -- recovering it
during tonight's incident required a one-off standalone Go program
reading the badger DB file directly.

Adds GET /users/persisted, returning every stored EmailAddress record.
Trades completeness (no online status, no Jellyfin username, no
last-active -- just enough to know who to email) for actually working
when Jellyfin is down. Same auth group (webAuth) as every other /users
route.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

⌂ 换网址