About a month ago, I had the privilege of returning to the University of Cambridge—not as a student this time, but as a guest lecturer. I was invited to speak at an Engineering Applications lecture, where I demonstrated an AI agent running on an Arm-based CPU. The experience was especially meaningful as it took place in the very same lecture theatre where I once sat as an engineering student.

Introduction

In the age of mobile-first experiences, running large language models (LLMs) directly on devices represents a powerful shift in how we approach AI applications. My latest project, LLM-on-iOS, demonstrates this paradigm by integrating the Gemma 2B language model into an iOS application using MediaPipe Tasks GenAI. This setup allows for efficient, on-device natural language processing and generation—entirely offline and privacy-preserving.

Introduction

In the summer of 2023, I had the incredible opportunity to work as a Machine Learning Research Intern at the Auto-ID Labs, KAIST in Daejeon, South Korea. Over the course of three months, I was immersed in cutting-edge AI research, deploying models on edge devices, and contributing to real-world applications. This blog post reflects on my technical journey, the challenges I faced, and the valuable lessons I learned.