GitHub/3x-ui
1
0
Fork 0
mirror of https://github.com/MHSanaei/3x-ui.git synced 2026-05-26 07:08:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(nodes): traffic-writer queue, full-mirror sync, WS event fixes

Browse Source 
- Traffic-writer single-consumer queue (web/service/traffic_writer.go)
  serialises every DB write that touches up/down/all_time/last_online
  (AddTraffic, SetRemoteTraffic, Reset*, UpdateClientTrafficByEmail) so
  overlapping goroutines can no longer clobber each other's column-scoped
  Updates with a stale tx.Save.

- DB pool: WAL + busy_timeout=10s + synchronous=NORMAL + _txlock=
  immediate, MaxOpenConns=8 / MaxIdleConns=4. The immediate-tx PRAGMA
  fixes residual "database is locked [0ms]" cases where deferred-tx
  writer-upgrade conflicts bypass busy_timeout.

- SetRemoteTraffic full-mirrors node-authoritative state into central:
  settings JSON, remark, listen, port, total, expiry, all_time, enable,
  plus per-client total/expiry/reset/all_time. Inbounds and
  client_traffics rows present on node but missing from central are
  created; rows missing from snap are deleted (with cascading
  client_traffics removal).

- NodeTrafficSyncJob detects structural changes from the mirror and
  broadcasts invalidate(inbounds) so open central UIs re-fetch via REST
  on node-side add/del/edit without manual refresh.

- XrayTrafficJob broadcasts invalidate(inbounds) when auto-disable flips
  client_traffics.enable so the per-client toggle reflects depletion
  without manual refresh.

- Frontend: inbounds page now subscribes to the BroadcastInbounds 'inbounds'
  WS event (full-list pushes from add/del/update controllers were silently
  dropped). Fixes invalidate payload field (dataType -> type). Restart-
  panel modal switched from Promise-wrap to onOk-only so Cancel actually
  cancels.

- Node files trimmed of stale prose-comments; cron cadence dropped
  10s -> 5s to match the inbounds page UX.

- README badges and Go module path bumped v2 -> v3 to match module rename.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
MHSanaei
2026-05-10 16:25:23 +02:00
parent 24cd271486
commit 8e7d215b4a
25 changed files with 559 additions and 639 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
web/runtime/local.go
View File

@@ -10,45 +10,22 @@ import (
"github.com/mhsanaei/3x-ui/v3/xray"
)
// LocalDeps wires the runtime to the panel's xray process and the
// service.XrayService restart trigger via callbacks. We use callbacks
// (not an interface to *service.XrayService) because the runtime
// package would otherwise cycle-import service.
type LocalDeps struct {
// APIPort returns the xray gRPC API port the local engine is
// currently listening on. Returns 0 when xray isn't running yet —
// callers should treat that as a transient error.
APIPort func() int
// SetNeedRestart trips the panel's "restart xray on next cron tick"
// flag. Mirrors how InboundController.addInbound calls
// xrayService.SetToNeedRestart() today.
APIPort func() int
SetNeedRestart func()
}
// Local implements Runtime against the panel's own xray process. Each
// call follows the existing inbound.go pattern: open a gRPC client,
// run one operation, close. Per-call init keeps the connection state
// scoped so a stuck call can't leak across operations.
type Local struct {
deps LocalDeps
// Serialise gRPC operations — xray's HandlerService isn't documented
// as concurrent-safe and the existing InboundService implicitly
// runs one op at a time per request. This matches that.
mu sync.Mutex
mu sync.Mutex
}
// NewLocal builds a Local runtime. deps.APIPort and deps.SetNeedRestart
// are required; callers that want a no-op restart can pass `func(){}`.
func NewLocal(deps LocalDeps) *Local {
return &Local{deps: deps}
}
func (l *Local) Name() string { return "local" }
// withAPI runs fn against a freshly-initialised XrayAPI client and
// guarantees Close() afterwards. Returns an error if the gRPC port
// isn't available yet (xray still starting / stopped).
func (l *Local) withAPI(fn func(api *xray.XrayAPI) error) error {
l.mu.Lock()
defer l.mu.Unlock()
@@ -82,17 +59,8 @@ func (l *Local) DelInbound(_ context.Context, ib *model.Inbound) error {
}
func (l *Local) UpdateInbound(ctx context.Context, oldIb, newIb *model.Inbound) error {
// xray-core has no in-place inbound update — drop and re-add.
// Matches what InboundService.UpdateInbound did inline.
if err := l.DelInbound(ctx, oldIb); err != nil {
// Best-effort: continue to AddInbound so a transient remove
// failure (e.g. inbound already gone) doesn't strand us. The
// caller's needRestart fallback will reconcile from config.
_ = err
}
_ = l.DelInbound(ctx, oldIb)
if !newIb.Enable {
// Disabled inbounds aren't pushed to xray; we already removed
// the old one above.
return nil
}
return l.AddInbound(ctx, newIb)
@@ -117,13 +85,6 @@ func (l *Local) RestartXray(_ context.Context) error {
return nil
}
// Reset methods are intentional no-ops for Local. The central DB UPDATE
// that runs in InboundService.Reset* before this call has already zeroed
// the counters that xray reads; on the next stats poll the gRPC service
// will pick up matching values. Pre-Phase-1 the panel never issued an
// xrayApi reset call here either — keeping the same shape avoids a
// behaviour change for single-panel users.
func (l *Local) ResetClientTraffic(_ context.Context, _ *model.Inbound, _ string) error {
return nil
}

39
web/runtime/manager.go
View File

@@ -8,14 +8,6 @@ import (
"github.com/mhsanaei/3x-ui/v3/database/model"
)
// Manager is the entry point for service code that needs a Runtime.
// One singleton lives in the package-level `manager` var, set at
// server bootstrap (web.go calls SetManager once). InboundService and
// friends read it via GetManager().
//
// Local runs forever; Remotes are built lazily per nodeID and cached.
// Cache invalidation runs on node Update/Delete (NodeService hooks
// InvalidateNode) so a token rotation surfaces the next call.
type Manager struct {
local Runtime
@@ -23,9 +15,6 @@ type Manager struct {
remotes map[int]*Remote
}
// NewManager wires the singleton with the deps Local needs. The runtime
// package can't import service so the caller (web.go) supplies the
// callbacks that bridge into XrayService.
func NewManager(localDeps LocalDeps) *Manager {
return &Manager{
local: NewLocal(localDeps),
@@ -33,10 +22,6 @@ func NewManager(localDeps LocalDeps) *Manager {
}
}
// RuntimeFor picks the right adapter for an inbound based on NodeID.
// Returns local when nodeID is nil; otherwise looks up the node row
// (or returns the cached Remote for it). The caller does not need to
// know which kind they got — that's the point of the abstraction.
func (m *Manager) RuntimeFor(nodeID *int) (Runtime, error) {
if nodeID == nil {
return m.local, nil
@@ -48,8 +33,6 @@ func (m *Manager) RuntimeFor(nodeID *int) (Runtime, error) {
}
m.mu.RUnlock()
// Cache miss — load the node row and build a Remote. We re-check
// under the write lock to avoid duplicate construction under load.
m.mu.Lock()
defer m.mu.Unlock()
if rt, ok := m.remotes[*nodeID]; ok {
@@ -67,16 +50,8 @@ func (m *Manager) RuntimeFor(nodeID *int) (Runtime, error) {
return rt, nil
}
// Local returns the singleton local runtime. Used by code that needs
// to operate on the panel's own xray regardless of which inbound it
// came from (e.g. on-demand restart from the UI).
func (m *Manager) Local() Runtime { return m.local }
// RemoteFor returns the Remote adapter for an already-loaded node row.
// Differs from RuntimeFor in two ways: it skips the DB lookup (caller
// hands in the node), and it returns the concrete *Remote so callers
// like NodeTrafficSyncJob can reach FetchTrafficSnapshot, which the
// Runtime interface doesn't expose.
func (m *Manager) RemoteFor(node *model.Node) (*Remote, error) {
if node == nil {
return nil, errors.New("node is nil")
@@ -98,18 +73,12 @@ func (m *Manager) RemoteFor(node *model.Node) (*Remote, error) {
return rt, nil
}
// InvalidateNode drops the cached Remote for nodeID so the next
// RuntimeFor call rebuilds it from the (possibly updated) node row.
// Called from NodeService.Update / Delete.
func (m *Manager) InvalidateNode(nodeID int) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.remotes, nodeID)
}
// loadNode reads a node row directly from the DB. Kept package-local
// to avoid pulling NodeService into the runtime — service depends on
// runtime, not the other way around.
func loadNode(id int) (*model.Node, error) {
db := database.GetDB()
n := &model.Node{}
@@ -119,25 +88,17 @@ func loadNode(id int) (*model.Node, error) {
return n, nil
}
// Singleton wiring -------------------------------------------------------
var (
managerMu sync.RWMutex
manager *Manager
)
// SetManager installs the process-wide Manager. web.go calls this once
// during NewServer. Tests can call it again with a stub.
func SetManager(m *Manager) {
managerMu.Lock()
defer managerMu.Unlock()
manager = m
}
// GetManager returns the installed Manager, or nil before SetManager
// has run. Callers should treat nil as "still booting" — the existing
// behaviour for code paths that only run on the local engine continues
// to work via a pre-wired fallback set up in init() below.
func GetManager() *Manager {
managerMu.RLock()
defer managerMu.RUnlock()

92
web/runtime/remote.go
View File

@@ -18,13 +18,8 @@ import (
"github.com/mhsanaei/3x-ui/v3/logger"
)
// remoteHTTPTimeout bounds a single remote API call. Generous enough for
// a slow node under load, short enough that a wedged remote doesn't
// block the central panel's UI thread for the user.
const remoteHTTPTimeout = 10 * time.Second
// remoteHTTPClient is shared so repeated calls to the same node reuse
// connections. Per-request timeouts are set via context.
var remoteHTTPClient = &http.Client{
Transport: &http.Transport{
MaxIdleConns: 64,
@@ -33,22 +28,12 @@ var remoteHTTPClient = &http.Client{
},
}
// envelope mirrors entity.Msg without depending on the entity package
// (avoids a cycle on the controller side that pulls in this runtime).
type envelope struct {
Success bool `json:"success"`
Msg string `json:"msg"`
Obj json.RawMessage `json:"obj"`
}
// Remote implements Runtime by calling the existing /panel/api/inbounds/*
// endpoints on a remote 3x-ui panel. The remote is authenticated as
// the central panel via its per-node Bearer token.
//
// remoteIDByTag caches the {tag → remote inbound id} mapping so the
// hot path (update/delete/addClient) avoids /list lookups. The cache
// is in-memory and rebuilt lazily on first miss after a process restart
// or InvalidateNode call.
type Remote struct {
node *model.Node
@@ -56,10 +41,6 @@ type Remote struct {
remoteIDByTag map[string]int
}
// NewRemote constructs a Remote runtime for one node. The node pointer
// is cached; callers that mutate node config (via NodeService.Update)
// must drop the runtime through Manager.InvalidateNode so a fresh one
// picks up the new fields.
func NewRemote(n *model.Node) *Remote {
return &Remote{
node: n,
@@ -69,8 +50,6 @@ func NewRemote(n *model.Node) *Remote {
func (r *Remote) Name() string { return "node:" + r.node.Name }
// baseURL composes the panel root for r.node, e.g. https://1.2.3.4:2053/
// Always ends in '/' so callers can append "panel/api/...".
func (r *Remote) baseURL() string {
bp := r.node.BasePath
if bp == "" {
@@ -82,13 +61,6 @@ func (r *Remote) baseURL() string {
return fmt.Sprintf("%s://%s:%d%s", r.node.Scheme, r.node.Address, r.node.Port, bp)
}
// do issues an HTTP request against the remote panel and decodes the
// entity.Msg envelope. Returns an error for transport failures, non-2xx
// responses, or {success:false} bodies.
//
// body may be nil. For application/x-www-form-urlencoded calls (the
// existing controllers bind via c.ShouldBind which prefers form-encoded)
// pass url.Values; for JSON pass any other type and we'll marshal it.
func (r *Remote) do(ctx context.Context, method, path string, body any) (*envelope, error) {
if r.node.ApiToken == "" {
return nil, errors.New("node has no API token configured")
@@ -102,7 +74,6 @@ func (r *Remote) do(ctx context.Context, method, path string, body any) (*envelo
)
switch b := body.(type) {
case nil:
// nothing
case url.Values:
reqBody = strings.NewReader(b.Encode())
contentType = "application/x-www-form-urlencoded"
@@ -151,10 +122,6 @@ func (r *Remote) do(ctx context.Context, method, path string, body any) (*envelo
return &env, nil
}
// resolveRemoteID returns the remote panel's local inbound ID for the
// given tag. Cache-backed; on miss it hits /panel/api/inbounds/list and
// repopulates the whole map (one-shot list is cheaper than per-tag
// lookups when several inbounds need resolving in sequence).
func (r *Remote) resolveRemoteID(ctx context.Context, tag string) (int, error) {
if id, ok := r.cacheGet(tag); ok {
return id, nil
@@ -187,9 +154,6 @@ func (r *Remote) cacheDel(tag string) {
delete(r.remoteIDByTag, tag)
}
// refreshRemoteIDs replaces the in-memory tag→id map with whatever the
// node currently has. Called on cache miss; also a useful recovery path
// when the remote panel is rebuilt or we get a "not found" on update.
func (r *Remote) refreshRemoteIDs(ctx context.Context) error {
env, err := r.do(ctx, http.MethodGet, "panel/api/inbounds/list", nil)
if err != nil {
@@ -216,17 +180,11 @@ func (r *Remote) refreshRemoteIDs(ctx context.Context) error {
}
func (r *Remote) AddInbound(ctx context.Context, ib *model.Inbound) error {
// Strip NodeID from the wire payload so the remote stores a "local"
// row from its own perspective. We also ship the full model.Inbound
// minus runtime metadata. Tag is preserved so central + remote agree
// on the identifier — relies on InboundController being patched to
// not overwrite a non-empty Tag.
payload := wireInbound(ib)
env, err := r.do(ctx, http.MethodPost, "panel/api/inbounds/add", payload)
if err != nil {
return err
}
// Response body contains the saved inbound (with the remote's Id).
var created struct {
Id int `json:"id"`
Tag string `json:"tag"`
@@ -242,9 +200,7 @@ func (r *Remote) AddInbound(ctx context.Context, ib *model.Inbound) error {
func (r *Remote) DelInbound(ctx context.Context, ib *model.Inbound) error {
id, err := r.resolveRemoteID(ctx, ib.Tag)
if err != nil {
// Already gone on remote — treat as success so a sync after a
// remote panel reset doesn't strand the central panel.
logger.Warning("remote DelInbound: tag", ib.Tag, "not found on", r.node.Name, "— treating as success")
logger.Warning("remote DelInbound: tag", ib.Tag, "not found on", r.node.Name)
return nil
}
if _, err := r.do(ctx, http.MethodPost, "panel/api/inbounds/del/"+strconv.Itoa(id), nil); err != nil {
@@ -255,21 +211,14 @@ func (r *Remote) DelInbound(ctx context.Context, ib *model.Inbound) error {
}
func (r *Remote) UpdateInbound(ctx context.Context, oldIb, newIb *model.Inbound) error {
// The remote's old row is keyed by oldIb.Tag (tags can change on
// edit if listen/port changed). We update by remote-id so the row
// keeps its identity even when its tag flips.
id, err := r.resolveRemoteID(ctx, oldIb.Tag)
if err != nil {
// Remote lost the row — fall back to add. This can happen if
// the node panel was reset; we'd rather end up with the inbound
// existing than fail the user's update.
return r.AddInbound(ctx, newIb)
}
payload := wireInbound(newIb)
if _, err := r.do(ctx, http.MethodPost, "panel/api/inbounds/update/"+strconv.Itoa(id), payload); err != nil {
return err
}
// Tag may have changed — remap the cache.
if oldIb.Tag != newIb.Tag {
r.cacheDel(oldIb.Tag)
}
@@ -277,18 +226,6 @@ func (r *Remote) UpdateInbound(ctx context.Context, oldIb, newIb *model.Inbound)
return nil
}
// AddUser pushes a single client into the remote inbound's settings JSON.
// We can't reuse the central panel's xrayApi.AddUser shape directly
// because the remote's HTTP endpoint expects {id, settings} where
// settings is a JSON string with a "clients":[...] array. The central
// panel's InboundService has already updated its own settings JSON
// before calling us, so we just ship the new full settings to the
// remote via /update — simpler than reconstructing the partial AddUser
// payload remote-side.
//
// Caller passes the full updated *model.Inbound on the same code path
// AddUser is called from in InboundService. To avoid changing the
// Runtime interface for that, AddUser/RemoveUser delegate to UpdateInbound.
func (r *Remote) AddUser(ctx context.Context, ib *model.Inbound, _ map[string]any) error {
return r.UpdateInbound(ctx, ib, ib)
}
@@ -305,8 +242,7 @@ func (r *Remote) RestartXray(ctx context.Context) error {
func (r *Remote) ResetClientTraffic(ctx context.Context, ib *model.Inbound, email string) error {
id, err := r.resolveRemoteID(ctx, ib.Tag)
if err != nil {
// Already gone on remote — central reset is enough.
logger.Warning("remote ResetClientTraffic: tag", ib.Tag, "not found on", r.node.Name, "— treating as success")
logger.Warning("remote ResetClientTraffic: tag", ib.Tag, "not found on", r.node.Name)
return nil
}
_, err = r.do(ctx, http.MethodPost,
@@ -318,7 +254,7 @@ func (r *Remote) ResetClientTraffic(ctx context.Context, ib *model.Inbound, emai
func (r *Remote) ResetInboundClientTraffics(ctx context.Context, ib *model.Inbound) error {
id, err := r.resolveRemoteID(ctx, ib.Tag)
if err != nil {
logger.Warning("remote ResetInboundClientTraffics: tag", ib.Tag, "not found on", r.node.Name, "— treating as success")
logger.Warning("remote ResetInboundClientTraffics: tag", ib.Tag, "not found on", r.node.Name)
return nil
}
_, err = r.do(ctx, http.MethodPost,
@@ -331,22 +267,12 @@ func (r *Remote) ResetAllTraffics(ctx context.Context) error {
return err
}
// TrafficSnapshot is what NodeTrafficSyncJob pulls from a remote node
// every cron tick. Inbounds carry absolute up/down/all_time + ClientStats
// (the same shape /panel/api/inbounds/list returns); the two map fields
// come from the dedicated /onlines and /lastOnline endpoints.
type TrafficSnapshot struct {
Inbounds []*model.Inbound
OnlineEmails []string
LastOnlineMap map[string]int64
}
// FetchTrafficSnapshot pulls the three pieces in series. Sequential is
// fine because the cron job already fans out across nodes — adding
// per-node parallelism on top would just thrash the remote.
//
// Not on the Runtime interface: only the sync job needs it, and Local
// has no equivalent (XrayTrafficJob already covers the local engine).
func (r *Remote) FetchTrafficSnapshot(ctx context.Context) (*TrafficSnapshot, error) {
snap := &TrafficSnapshot{LastOnlineMap: map[string]int64{}}
@@ -360,9 +286,6 @@ func (r *Remote) FetchTrafficSnapshot(ctx context.Context) (*TrafficSnapshot, er
envOnlines, err := r.do(ctx, http.MethodPost, "panel/api/inbounds/onlines", nil)
if err != nil {
// Onlines/lastOnline are nice-to-have. A failure here shouldn't
// invalidate the inbound counter merge — log and continue with
// empty values, the next tick may succeed.
logger.Warning("remote", r.node.Name, "onlines fetch failed:", err)
} else if len(envOnlines.Obj) > 0 {
_ = json.Unmarshal(envOnlines.Obj, &snap.OnlineEmails)
@@ -378,17 +301,8 @@ func (r *Remote) FetchTrafficSnapshot(ctx context.Context) (*TrafficSnapshot, er
return snap, nil
}
// wireInbound builds the request body for /panel/api/inbounds/add and
// /update. Mirrors the form fields the existing InboundController
// expects via c.ShouldBind — we use form-encoded to match exactly.
//
// We deliberately omit Id (remote assigns its own), UserId (remote's
// fallback user takes over), NodeID (the remote sees itself as local),
// and ClientStats (those are joined-table data the remote rebuilds).
func wireInbound(ib *model.Inbound) url.Values {
v := url.Values{}
v.Set("up", strconv.FormatInt(ib.Up, 10))
v.Set("down", strconv.FormatInt(ib.Down, 10))
v.Set("total", strconv.FormatInt(ib.Total, 10))
v.Set("remark", ib.Remark)
v.Set("enable", strconv.FormatBool(ib.Enable))

42
web/runtime/runtime.go
View File

@@ -1,12 +1,3 @@
// Package runtime abstracts the live xray engine that an inbound's
// configuration is shipped to. Two implementations exist: Local talks
// to the panel's own xray via gRPC (the original behaviour); Remote
// talks to another 3x-ui panel's HTTP API as a managed Node.
//
// InboundService picks a Runtime per-inbound based on Inbound.NodeID.
// The point of the abstraction is to keep `if node != nil` checks out
// of the service code as Phase 2/3 features (traffic sync, subscription
// per-node) build on top.
package runtime
import (
@@ -15,52 +6,19 @@ import (
"github.com/mhsanaei/3x-ui/v3/database/model"
)
// Runtime is the live-engine adapter for one inbound's worth of
// operations. Implementations must be safe for concurrent use — the
// service layer does not synchronise calls.
type Runtime interface {
// Name identifies the adapter in logs ("local", "node:<name>").
Name() string
// AddInbound deploys an inbound to the engine. The Tag field on ib
// is treated as the source of truth for identifying the inbound on
// the remote side; Local ignores it.
AddInbound(ctx context.Context, ib *model.Inbound) error
// DelInbound removes the inbound identified by ib.Tag.
DelInbound(ctx context.Context, ib *model.Inbound) error
// UpdateInbound replaces the existing inbound with newIb. oldIb
// carries the previous config so the adapter can compute a minimal
// diff (Local: drop+add by tag; Remote: HTTP update by remote-id).
UpdateInbound(ctx context.Context, oldIb, newIb *model.Inbound) error
// AddUser hot-adds a client to the inbound identified by ib.Tag.
// userMap matches the shape that xray.XrayAPI.AddUser already takes
// — keys: email, id, password, auth, security, flow, cipher.
AddUser(ctx context.Context, ib *model.Inbound, userMap map[string]any) error
// RemoveUser hot-removes the client by email from ib's inbound.
RemoveUser(ctx context.Context, ib *model.Inbound, email string) error
// RestartXray asks the engine to fully restart. For Local this just
// flips the SetToNeedRestart flag and lets the cron pick it up; for
// Remote it issues an HTTP POST to /panel/api/server/restartXrayService.
RestartXray(ctx context.Context) error
// ResetClientTraffic zeros the up/down counters for one client on the
// engine. Local: no-op — the central DB UPDATE that runs before this
// call is sufficient, and xray's gRPC stats counter resets on the next
// poll. Remote: HTTP POST so the next traffic sync doesn't pull the
// pre-reset absolute back from the node.
ResetClientTraffic(ctx context.Context, ib *model.Inbound, email string) error
// ResetInboundClientTraffics zeros every client of one inbound. Same
// Local/Remote split as ResetClientTraffic.
ResetInboundClientTraffics(ctx context.Context, ib *model.Inbound) error
// ResetAllTraffics zeros every inbound counter on the engine. Used by
// the panel-wide "reset all traffic" action; called once per affected
// node so that nodes with no inbounds for the current panel are skipped.
ResetAllTraffics(ctx context.Context) error
}