End-to-End AI Systems: From Dataset Construction to Stabilisation & Deployment
This week marks the long-awaited return of our student projects series, where we showcase the remarkable work our students have been building. This time, we sat down with David, an AI enthusiast and recent Master of Data Science graduate, to share his project and the journey behind it. His story is a reminder of how true motivation and perseverance can open unexpected doors, and we hope it will encourage others to pursue their own goals in the same spirit.
About Zhengyang Li (David)
Q: Can you tell us a bit about yourself and what you enjoy doing in your free time?
I recently graduated from the Master of Data Science at the University of Adelaide. Before that, I completed a Master of Physics, which shaped the way I think about quantitative problems and systems. Over the past few years, I’ve built up close to three years of industry experience working across AIOps and DevOps, where my work sits at the intersection of AI systems, data engineering, and infrastructure.
My technical focus includes building AI pipelines end-to-end high-concurrency data crawlers, data cleaning workflows, model training setups, image-to-text extraction, text embedding, and vector databases. On the operations side, I manage Linux systems, containerised environments, and AI-related infrastructure. I’ve set up and maintained my own AI workstation and frequently work with Docker, Supabase, Qdrant, MariaDB, and llama.cpp for different production tasks.
Outside of work, I enjoy building side projects, experimenting with system automation, and maintaining my self-hosted blog on homelab. To unwind, I usually go for long walks, cook, or read about science and technology.
What inspired your project?
Q: Can you briefly explain what your research project to non-technical readers?
My project explores how to make AI systems safer and more reliable, especially when they are used in situations where accuracy and fairness matter. Part of my work focuses on understanding how AI models for time-series data can be deliberately misled through small, carefully designed disturbances, and how we can strengthen these models so they remain stable even under such attacks. I also build practical systems that put these ideas into real use, for example, developing an anti-cheat exam platform that monitors digital exam environments and reduces the risk of misuse of AI tools during assessments. The overall aim is to make AI behave more responsibly: understanding its weaknesses, improving its robustness, and designing systems that prevent inappropriate usage. This work has been carried out over the past two to three years through research, coursework, and industry-oriented development.
Q: What problems or knowledge gap inspired your project?
My inspiration came from noticing a simple but important problem: AI systems are becoming more capable, yet many of them can be unexpectedly fragile or easily misused. When I first worked with time-series models, I realised that very small, carefully designed changes in the input could make a model completely misinterpret the data. This discovery pushed me to look deeper into adversarial attacks and the broader question of how we can make AI models more dependable in real-world settings.
At the same time, seeing how quickly AI tools entered everyday student life made me aware of another gap: we often rely on AI before we fully understand its risks. This motivated me to build a practical anti-cheat system that helps create fairer exam environments and prevents the irresponsible use of AI during assessments.
Together, these experiences shaped my interest in safe and responsible AI. The inspiration wasn’t a single moment, but a growing awareness that building smarter systems also means understanding their weaknesses and designing ways to use them wisely.
Approaches, Challenges and Findings
Q: Can you walk me through your technical approach?
For one of my recent projects, the data mainly came from open-source audio datasets, because obtaining commercial-grade audio data with full licensing is often challenging. To fill those gaps I also developed custom crawlers that collected additional video&audio signals under ethical and compliant usage scenarios.
In terms of modelling and tooling, I leveraged methods drawn from my research on time-series classification, adversarial robustness and AI systems design. The approach comprised:
Pre-processing the audio signals (e.g., normalisation, trimming silence, converting to spectrogram or embedding vectors)
Using a combination of machine learning models and pipelines: for example feature-extraction → embedding → classifier; in some cases applying vector-database retrieval for similarity search and robustness checks
Deploying the system in a production-style environment: managing the Linux servers, containerising with Docker, orchestrating services, working with databases like MariaDB and vector stores such as Qdrant and llama.cpp for specialised inference tasks
Stress-testing and hardening: since the system was exposed to real-world workflows, I had to account for hardware-compatibility issues (GPUs, drivers, container runtime, etc.), build monitoring/alerting for performance and failures, and ensure the “AI in production” scenario matches the “AI in lab” scenario.
Q: What were your key findings and impact of your work?
One of the most important findings from my work is that AI models, especially those used for time-series data, are far more sensitive to small manipulations than we normally assume. Even tiny changes that are invisible to humans can cause a model to make completely different decisions. Through my research, I was able to show which types of disturbances are most harmful and which defence methods, such as self-ensembling, smoothing, or specific data-augmentation strategies—actually make a model more stable. This helps us understand not only how AI fails, but also how we can design it to fail less often.
Another outcome comes from applying these ideas to real systems. When building AI-driven tools such as an academic integrity monitoring platform, I learned that integrating AI into real workflows always reveals new challenges: unreliable data sources, hardware incompatibilities, performance bottlenecks, and the risk of users misusing AI tools. By solving these problems in practice, the project showed that “responsible use of AI” is not just a research topic, it shapes how systems are designed, deployed, and controlled.
There is still plenty of room for improvement. Models can be made more transparent, defence strategies can be more adaptive, and real-world systems can include stronger safeguards to prevent misuse. But the key impact of this work is in demonstrating that safer AI is achievable when we combine scientific analysis with practical system design. These insights can be applied to many areas—healthcare monitoring, finance, education, or any setting where AI decisions need to be both accurate and trustworthy.
Q: What was one of the biggest challenges you faced during your work?
One of the biggest challenges was that moving from research to production required re-thinking the code and architecture: what works in a paper (prototype) often isn’t robust in a live system with many users or high concurrency. For example, I ran into compatibility issues around GPU driver stacks when switching to the open GPU-stack (such as ROCm on AMD hardware), which required deep debugging and custom bundling of libraries.
To overcome these, I created a reproducible container image, set up continuous monitoring of hardware metrics, and documented a checklist for deployment transitions. Another challenge was data sensitivity and licensing: I ensured all data were used under permissible licenses, and for prototypes clearly marked commercial-use vs research, use to avoid misuse.
Growth & Development
Q: How did this project help you grow or develop your skills for your career?
Working on this project gave me a more realistic understanding of what it means to build reliable AI systems. The development process was not always straightforward: models behaved differently once they moved out of controlled research environments, data quality changed over time, and even hardware or driver updates sometimes broke the entire workflow. These challenges pushed me to think more carefully about system design, documentation, testing, and long-term maintainability.
This experience shaped my career in two ways. Technically, I became much more comfortable working across the full stack of AI systems, from data collection to model robustness analysis to deployment and monitoring. I also learned the importance of anticipating misuse and designing safeguards early, which is central to responsible AI practice. On the soft-skill side, I developed more patience and discipline in troubleshooting, as well as clearer communication when explaining technical risks to non-technical collaborators.
Q: What would you do differently if you have more time or if you could start the project again?
If I had the chance to go back to the beginning, I wouldn’t change any part of the process. The challenges, mistakes and unexpected turns were all essential in shaping how I think about AI systems today. Even the parts that felt inefficient or confusing at the time ended up teaching me something important, about problem-solving, responsibility, and how real systems behave outside ideal conditions. So rather than giving my past self technical advice, I would probably just say: stay curious, trust your direction, and keep going.
Going forward, I hope to extend this work in two directions: strengthening robustness evaluation for real-world data, and refining the system design so that it can support larger workloads with better transparency and safety controls. The overarching goal is to continue building AI that is not only powerful, but also reliable and responsibly deployed.
Advice for new Generations
“I have no way. I began with what I was used to, grew up with my nature, and let things come to completion with fate. I go under with the swirls and come out with the eddies, following along the way the water goes and never thinking about myself. That's how I can stay afloat.”
- Chuang Zhu -
Q: What advice would you give to the next generation of students?
My advice to the next generation is simple: move with curiosity, not with fear. Too often we aim for “perfect projects” and avoid the messy middle where real growth happens. If you lean into the process, the uncertain steps, the unexpected outcomes, you’ll learn more and build things that matter.
In practical terms:
Start your project by asking why, not how. When the why is rooted in real interest, the how becomes natural.
Embrace small missteps. They are the soil where insights grow.
When you communicate with supervisors or teammates, be honest about where you’re stuck and where you’re exploring. That openness builds trust, which matters more than appearing flawless.
Work steadily, not always wildly. Sticking to something long enough builds depth.
And finally: remember that technology serves people and purpose, not the other way around. Let your work flow from your interest, not from a checklist.
In a way, this is my attempt to live a bit of Zhuangzi’s idea: be like a tree growing where it lands, adapting to wind and sun, rather than forcing itself into a shape. If you trust the path under your feet, you’ll find your direction.
Looking into the future
Q: Can you tell me a bit about your future plans?
I plan to continue working in the AI and systems space, especially in areas where machine learning meets real-world infrastructure. The combination of AIOps, DevOps and model robustness has shown me that building reliable AI is just as important as building powerful AI, and this is a direction I want to keep exploring.
As for the next five or ten years, I don’t lock myself into a fixed destination. If there is anything this field has taught me, it’s that technology changes faster than our plans do. I hope to grow into someone who can design and maintain trustworthy AI systems, whether that means working as an engineer, leading a technical team, or contributing to open-source tools that make AI safer and more accessible. I’m more interested in the path than in predicting an exact title.
AI expands opportunity for those who learn and adapt, but it also threatens to replace roles that don’t evolve.
Q: How do you see AI in the future as an opportunity and a challenge?
While I believe the field will continue to grow, I also see AI as a double-edged sword. For people who are curious and willing to learn, AI opens doors that were previously inaccessible: they can build, explore and create at a scale that used to require entire teams. But for those who prefer to rely only on repetitive routines, the future may feel more challenging, because automation will gradually replace work that doesn’t evolve.
At the same time, AI also narrows certain gaps. The cost of accessing knowledge, tools and high-quality education is becoming lower than ever. Someone’s background matters less than their willingness to engage, learn and adapt. In that sense, AI amplifies both sides: it widens the distance between different attitudes toward learning, yet shrinks the distance between different levels of opportunity.
My hope is to contribute to this space in a way that keeps the benefits accessible while ensuring the technology is used responsibly. The real world is a chaotic system, and I want to help build systems that remain reliable within that complexity.
Thank you David for the interview, and I wish you all the best in your life and career ahead. We look forward to having your project showcased in the future.
