UWSpace

UWSpace is the University of Waterloo’s institutional repository for the free, secure, and long-term home of research produced by faculty, students, and staff.

Depositing Theses/Dissertations or Research to UWSpace

Are you a Graduate Student depositing your thesis to UWSpace? See our Thesis Deposit Help and UWSpace Thesis FAQ pages to learn more.

Are you a Faculty or Staff member depositing research to UWSpace? See our Waterloo Research Deposit Help and Self-Archiving pages to learn more.

Photo by Waterloo staff

Recent Submissions

  • Item type: Item ,
    Fully non-zero matrices and generators of full algebras of operators
    (Elsevier, 2027) Marcoux, Laurent W.; Radjavi, Heydar; Yahaghi, Bamdad R.; Zhang, Yuanhang
    In a wide variety of fields, every non-scalar matrix M is similar to a matrix all of whose entries are non-zero. We give extensions of this result, including some in the division ring context. We then apply these results to the question of which pairs of operators generate the algebra of all operators on finite- and infinite-dimensional spaces. In the case of complex, separable infinite-dimensional Hilbert space, this is intimately related to the unsolved Invariant Subspace Problem. Finally, linear independence of subsets of arbitrary TVS's and generators of arbitrary TA's are shown to be stable in the sense of the theorems presented.
  • Item type: Item ,
    On commutators of square-zero Hilbert space operators
    (Cambridge University Press, 2026) Marcoux, Laurent W.; Radjavi, Heydar; Zhang, Yuanhang
    Let H be a complex, separable Hilbert space, and set c(nil2) = {MN −NM : N, M ∈ B(H), M2 = 0 = N 2 }. When dim H is finite, we characterise the set c(nil2) and its norm-closure clos(c(nil2)). In the infinite dimensional setting, we characterise the intersection of clos(c(nil2)) with the set of biquasitriangular operators, and we exhibit an index obstruction to belonging to clos(c(nil2)).
  • Item type: Item ,
    All unitary operators are tricycles, and most are bicycles
    (Wiley, 2026) Marcoux, Laurent W.; Omladic, Matjaz; Radjavi, Heydar; Zhang, Yuanhang
    Let H be a complex, separable Hilbert space (of finite or infinite dimensino), and let U(H) denote the group of unitary operators on H. In the finite-dimensional setting, every unitary operator of determinant one can be expressed as the product of two operators, each unitarily equivalent to the n x n unitary cycle. In the infinite-dimensional setting, we prove that every unitary operator U is a product of three operators, each unitarily equivalent to the bilateral shift, and if the spectrum of U has non-zero Lebesque measure, then U is a product of two operators, each unitarily equivalent to the bilateral shift.
  • Item type: Item ,
    On products of symmetries acting on Hilbert spaces
    (Institute of Mathematics, 2026) Marcoux, Laurent W.; Radjavi, Heydar; Zhang, Yuanhang
    Let H be a complex, separable Hilbert space (of finite or infinite dimension), and let U(H) denote the group of unitary operators on H. A symmetry is, by definition, a unitary operator J with J 2 = I. Denote by Symk(H) the subset of U(H) consisting of those operators expressible as a product of k symmetries. It is known that U(H) = Sym4(H) if dim H = ∞, while the only additional condition in finite dimensions is that the determinant be ±1. Of all the sets Symk(H) with k ∈ {1, 2, 3, 4}, the case k = 3 has been the most stubborn to characterise. Among other things, we investigate which elements of Sym3(H) possess exactly two eigenvalues in the finite-dimensional setting. We also consider the problem: when is the unitary orbit of an operator T, i.e., the set {U ∗T U : U ∈ U(H)} the same as its Symk-orbit, i.e., the set {U ∗T U : U ∈ Symk(H)}? Clearly, the cases of interest are when k ≤ 3.
  • Item type: Item ,
    Enhancing Neural Network Efficiency: Novel Approaches to Knowledge Distillation
    (University of Waterloo, 2026-11-18) Jafari, Aref
    Deep neural networks have achieved remarkable success across natural language processing and computer vision. However, the growing scale of these models has led to prohibitive computational and memory requirements, making their deployment on resource-limited devices increasingly challenging. Knowledge Distillation (KD) has emerged as a powerful model compression technique that enables a smaller student network to learn from a larger, more expressive teacher model. Despite its effectiveness, classical KD faces several fundamental challenges: (i) the capacity gap problem, where the student struggles to learn from an overly large teacher; (ii) the checkpoint search problem, where the best-trained teacher is not necessarily the optimal one for distillation; (iii) the noisy teacher problem, where unreliable teacher outputs can degrade the student's learning; and (iv) the generative distribution-mismatch problem (of which exposure bias is one facet), where students trained on a fixed set of teacher-generated sequences behave very differently when generating their own outputs at inference time. This thesis proposes five novel methods that address these challenges across three complementary paradigms: logit distillation, data distillation, and divergence-aware distillation. In the logit distillation paradigm, we first propose Annealing-KD, which tackles the capacity gap by applying a dynamic temperature function exclusively to the teacher's logits, constructing an entropy-based curriculum that gradually exposes the student to increasingly sharp targets. Building on known properties of the temperature-scaled softmax, we give a self-contained analysis showing that the entropy of the teacher's temperature-scaled distribution increases monotonically with temperature (strictly, whenever the logits are not all equal), which provides a principled mechanism for controlling the sharpness of the supervisory signal. Second, we propose Progressive Knowledge Distillation (Pro-KD), which addresses both the capacity gap and the checkpoint search problem by having the student follow the teacher's entire training trajectory rather than distilling from a single, fully converged checkpoint. Combined with an adaptive temperature schedule, Pro-KD creates a doubly smooth curriculum that eliminates the need for costly checkpoint searches. Third, we propose Continuation-KD, which generalizes Annealing-KD by replacing its rigid two-stage structure with a single scheduled objective inspired by the continuation method from optimization theory. Continuation-KD's mechanism is an annealed hinge-loss margin: once the student's residual to the scaled teacher falls below the margin, the distillation term is exactly flat there and produces no gradient, so noise-level differences from the teacher are not transferred to the student. We give two exact identities for the annealed objective (one for the gradient, one for the loss Hessian) that isolate the teacher's contribution in a single term scaled linearly by the annealing factor and make precise what the hinge removes. We are explicit that the step from this structure to improved optimization and generalization is not established by our experiments. All three methods consistently outperform the vanilla-KD and TAKD baselines on the GLUE benchmark, CIFAR-10, and CIFAR-100. Moving beyond white-box distillation, we apply the data distillation paradigm to citation-grounded question answering. Using ChatGPT as a black-box teacher, we synthesize a training corpus of citation-enriched answers and fine-tune two compact student models: FIDCiter (3B parameters) and LLaMACiter (13B parameters). LLaMACiter exceeds the teacher on human-rated informativeness across all four evaluation sets and FIDCiter on two of the four, and FIDCiter matches or exceeds it on in-domain citation quality, while both use much shorter instruction templates; the teacher retains an edge on citation quality out of domain. Finally, we propose Maximum Divergence Knowledge Distillation (MDKD) for autoregressive language model distillation. MDKD uses a rejection-sampling mechanism to preferentially select tokens where the student underestimates the teacher, constructing training sequences concentrated in regions where the student underweights teacher-supported tokens. We prove that the stochastic acceptance branch samples exactly from the positive teacher–student residual, with acceptance probability equal to their total variation distance. This active, divergence-aware approach targets the distributional-coverage aspect of the training–inference mismatch in sequence-level distillation; because its training prefixes are teacher-generated, direct on-policy mitigation of exposure bias is left to future work. Collectively, the five proposed methods trace a progression from passive distillation with static objectives to active, adaptive paradigms that tailor the supervisory signal to the student's current learning state. By addressing the capacity gap, checkpoint search, noisy teacher, and generative distribution-mismatch problems through complementary mechanisms, this thesis advances the design of efficient, high-performing models that bridge the gap between large-scale deep learning and real-world deployment constraints.