Agent is not a new concept, and the importance of agent in the AI era has been well recoganized. An agent can be viewed as a neural-symbolic approach, as discussed here. But I prefer to view agent as intelligence augmented by tools and their associated environments. Just as human’s capability is greatly enhanced by tools, AI’s capability grows significantly with the help of tools. Our study suggests that AI’s math reasoning can be significantly improved with the help of an MCTS-based, code augmented tool (Guan et al., 2025)(Shang et al., 2025), or a multi-agent system through a generation-discrimination process (Qi et al., 2025). More recently, we discover an agentic flow resembles the old DSP (draft, sketch, and prove) paradigm where general LLMs’ math reasoning capability can compete with LLMs trained specifically for math, even without any training (Cao et al., 2025). Based on this insight, we discover a new agentic reasoning paradigm, vibe reasoning(Wu et al., 2025), where AI reasons under the problem-agnostic guidance from human. Vibe reasoning solves IMO’25 P6, currently beyond the reach of frontier AI.
AI compiler translates neural network into low level device code, e.g., CUDA. It plays a critical role to ensure the efficient scaling of neural network.
The future of AI depends on whether we can design next generation hardware that better supports the scaling laws. At some point of time, AI model architecture will even be influenced by the design decision on AI hardware. The codesign of AI and hardware will become norm in the future.
I believe a neuro-symbolic approach is the only way to realize truthworthy reasoning, thus achieving true intelligence. It is probably the only hope to bridge the seemingly insurmountable gap between “correlation” and “causality”.
Vector store is an important storage system in the AI era. I believe it not only serves as an cache used to retrieve computation results from neural networks, but also becomes (or will become) a fundamental component at the core of neural models. For example, we realized that attention, a fundamental mechanism in the Transform-like neural architecture, can be viewed as vector index traversal (Liu et al., 2025). This makes the computation of sparse attention much more efficient. In the case of LLMs with a long context window, the benefit can be of one or multiple orders of magnitude. Recently, we have realized the connection between vector store and the long-term memory, similar to human’s hippocampus. We observe that long-term memory plays a critical role in pushing the boundary of human intelligence. We will share more soon.
AI workload differs significantly from conventional cloud workload (e.g., big data OLAP or OLTP workload). In early 2017, I began to look into this problem and tried to understand the implication. With my colleagues, we investigated a massive amount of AI workloads in Philly, Microsoft’s early GPU cluster management system designed for deep learning training. We shared our findings in (Jeon et al., 2019). We explained our thoughts on the scheduling primitives for training jobs (Xiao et al., 2018), and emphasized on the importance of topology aware scheduling (Zhao et al., 2020) in the AI era. Meanwhile, we also discovered several interesting opportunities in the coexistence of gaming and training workloads (Zhang et al., 2022), codesign of caching and scheduling (Zhao et al., 2023), and elastic training (Gu et al., 2023).
In the past, I was interested in Graph due to its complex structure and the potential in application domains like social network, web, and bio-science. I developed several graph engines. Specifically, Kineograph (Cheng et al., 2012) is designed for OLAP and OLTP on fast changing graphs, Chronos (Han et al., 2014) is a system for temporal graph analytics, and GraM (Wu et al., 2015) is a high-performance graph engine that set a new speed record for trillion-scale graph analytics. GraM has been used to analyze the large-scale web data and internal Ads data of the Microsoft Bing service.