Xiyang Zhang
Publications
ALIVE: Warnings Before Exclusion in Budgeted Multi-Source Learning
A routing decision can be revised at the next transaction, but a latched source exclusion persists across later decisions. We ask what evidence should authorize these unequal-persistence actions when finite-population auditing and learning share a budget. ALIVE (Action-Layered Intervention via Evidence) is an auditable control layer: one randomized without-replacement prefix supplies cached evidence, heuristic warnings drive non-latching floor-bounded routing, and only two fresh simultaneous certificate separations may latch an exclusion request subject to capacity-feasible activation. Conditional on fixed support and labels under an ideal uniform audit permutation, any predictable controller preserving this interface inherits an anytime familywise bound of δon acting against a source that fails the pre-fixed absolute or relative strict-majority-disagreement predicate. With a published known-size, all-strict-majority PPR engine, median evidence count fell from 304 to 96 identities in e40 and from 171 to 62 in e60, while both engines used 48 in e80. In the matched CIFAR controller, the persistent-action layer added +0.1935 accuracy-AUBC percentage points over routing-only in all ten paired seed clusters. The +0.1954-point full-system contrast against CBR was also positive but did not meet the predeclared multiplicity-adjusted criterion (conditional Holm-adjusted sign-flip reference value =.097656). On a fixed natural panel, exploratory PPR used a median closure prefix of 95 rather than 105 for exploratory Serfling/FPC, but still exposed 88.0% of the panel and had no downstream task. Together these results map a restraint--power--cost--utility boundary: the action contract controls a defined persistent decision, while net value depends on evidence margin, audit cost, and budget regime.
Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation
Fine-tuning large language models (LLMs) for specialized domains often necessitates a trade-off between acquiring domain expertise and retaining general reasoning capabilities, a phenomenon known as catastrophic forgetting. Existing remedies face a dichotomy: gradient surgery methods offer geometric safety but incur prohibitive computational costs via online projections, while efficient data selection approaches reduce overhead but remain blind to conflict-inducing gradient directions. In this paper, we propose Orthogonal Gradient Selection (OGS), a data-centric method that harmonizes domain performance, general capability retention, and training efficiency. OGS shifts the geometric insights of gradient projection from the optimizer to the data selection stage by treating data selection as a constrained decision-making process. By leveraging a lightweight Navigator model and reinforcement learning techniques, OGS dynamically identifies training samples whose gradients are orthogonal to a general-knowledge anchor. This approach ensures naturally safe updates for target models without modifying the optimizer or incurring runtime projection costs. Experiments across medical, legal, and financial domains demonstrate that OGS achieves excellent results, significantly improving domain performance and training efficiency while maintaining or even enhancing performance on general tasks such as GSM8K.