What follows here are the (semi-technical) chronicles of the research I've done so far, intermingled with where I would like my work to go.

Machine Learning

My work on the theory of Deep Learning has so far mostly focused on the question of generalization and the apparent absence of "over-fitting" by Deep Neural Networks, defined as follows: the expected error does not get worse when increasing the number of neurons or of iterations of gradient descent. The results of the work I'm part of suggest that stochastic gradient descent algorithms generically converge to global minima and that simple variations of GD converge to the minimum norm, maximum margin solution, thus providing an implicit mechanism for complexity control.  More recently, I have been working on deriving the empirically seen phenomenon of Neural Collapse, which suggests that in late stage of training dynamics of learning in deep networks greatly simplify, with the classifiers and the last layer features taking on a simple self-dual geometrical structure.

A second major direction of my research has been the problem of adversarial attacks on neural networks. Being surrounded by neuroscientists at MIT, I have been working on incorporating ideas from biology as potential defenses against adversaries. CNNs are currently the best models of primate vision, but they are significantly more susceptible to small perturbations - why is that? Is there something missing from these models, or are they completely wrong? So far, we have managed to show that including simple mechanisms of non-uniform sampling in the retina and eccentricity dependence in the cortex do lead to significant improvements in robustness, but there is still a big gap between humans and these systems.

I have also been working with my students on combining the neural network approach together with the work in the field of program synthesis to attempt tackling more abstract, rule-based symbolic reasoning. Such a combination is crucial, as standard neural networks seem to fail spectacularly at learning exact rules. The introduction of the Abstraction and Reasoning Corpus dataset by Chollet has been a perfect testbed for some of our ideas

 

Theoretical Physics

I started my PhD research by investigating Relative Locality - a phenomenological model of quantum gravity induced modifications to relativistic dynamics of point particles. One of the motivations behind it that I found very enticing is that when we perform experiments, we never actually measure distances, but rather we infer them from energy and momentum of quantum particles we emit and absorb and the timing of these events. This, together with the lesson from lower-dimensional models that gravitational effects modify the standard relativistic dispersion relations, motivate us to study particle dynamics on momentum spaces with non-trivial geometry. In my first work in Relative Locality, I showed that allowing such curvature can have the consequence of breaking global momentum conservation for a single propagating particle. I followed this with a construction of a curved momentum space that preserved full Lorentz symmetry. The non-commutative structure of the result turned out to be related to the discrete Snyder spacetime.

 Following my work in phenomenology, I decided to shift my focus to Spin Foam models - a proposal for path integral quantization of General Relativity. The central insight in Spin Foams is that gravity can be described by imposing so-called simplicity constraints onto a topological field theory (BF theory). Using a newly developed spinor representation of spin networks, together with my collaborators, we have discovered a simpler way of imposing these constraints. This allowed us to obtain the first analytical results for behavior of transition amplitudes in 4D quantum gravity under changes of triangulation (more technically, we evaluated the 4-dimensional Pachner moves). This work gives the hope of analytically studying the non-perturbative renormalization of 4D quantum gravity, which is one of the projects I am currently working on.

 Another direction I am investigating currently concerns the nature of time in quantum gravitational scenarios. The question that I am curious about is how would one go about operationally defining time intervals in extreme gravity regions. Our current relational definitions that are used in atomic clocks are only valid due to the existence of bound states - these however are not stable in extreme situations.

 

Mixed Reality

I have spent two summers working at Microsoft Research under Jaron Lanier on mixed reality applications to physics and mathematics research. The first time, I was part of the COMRADRE (Center of Mixed Reality Advanced Development and Research) lab in Redmond, WA. There, we worked on novel applications of mixed reality on hacked together headsets called "Reality Mashers". Apart from smaller applications, like visualizing a hypercube that was superimposed onto a physical cube that a user could rotate, I prototyped an environment for visualizing mathematical equations and performing operations on them using gestures and voice input.

My second internship took place in Mountain View, CA, where I pushed the idea from my previous work further. This time I switched my work to the HoloLens platform and created a multi-user collaborative platform for mathematical research. I explored new ways of interacting with mathematical objects. For example, transformations were represented by virtual magnifying lenses. If a user looks through the lens at a data set, the rendered image is modified by the applied transformation. The interesting aspect of this is that several lenses can be put in sequence, to obtain compositions, or can be reversed to get the inverse operation. This allows multiple users to explore a meta-structure of related mathematical expressions, a task that gets confusing in a 2D medium.