This week I welcome Dr. Andrew Clark, Co-founder & CTO of Monitaur, a trusted domain expert on the topic of machine learning, auditing and assurance; and Sid Mangalik, Research Scientist at Monitaur and PhD student at Stony Brook University. I discovered Andrew and Sid's new podcast show, The AI Fundamentalists Podcast. I very much enjoyed their lively episode on Synthetic Data & AI, and am delighted to introduce them to my audience of privacy engineers.
In our conversation, we explore why data scientists must stress test their model validations, especially for consequential systems that affect human safety and reliability. In fact, we have much to learn from the aerospace engineering field who has been using ML/AI since the 1960s. We discuss the best and worst use cases for using synthetic data'; problems with LLM-generated synthetic data; what can go wrong when your AI models lack diversity; how to build fair, performant systems; & synthetic data techniques for use with AI.
Topics Covered:
- What inspired Andrew to found Monitaur and focus on AI governance
- Sid’s career path and his current PhD focus on NLP
- What motivated Andrew & Sid to launch their podcast, The AI Fundamentalists
- Defining 'synthetic data' & why academia takes a more rigorous approach to synthetic data than industry
- Whether the output of LLMs are synthetic data & the problem with training LLM base models with this data
- The best and worst 'synthetic data' use cases for ML/AI
- Why the 'quality' of input data is so important when training AI models
- Thoughts on OpenAI's announcement that it will use LLM-generated synthetic data; and critique of OpenAI's approach, the AI hype machine, and the problems with 'growth hacking' corner-cutting
- The importance of diversity when training AI models; using 'multi-objective modeling' for building fair & performant systems
- Andrew unpacks the "fairness through unawareness fallacy"
- How 'randomized data' differs from 'synthetic data'
- 4 techniques for using synthetic data with ML/AI: 1) the Monte Carlo method; 2) Latin hypercube sampling; 3) gaussian copulas; & 4) random walking
- What excites Andrew & Sid about synthetic data and how it will be used with AI in the future
Resources Mentioned:
Guest Info:
Send us a text
Privado.ai
Privacy assurance at the speed of product development. Get instant visibility w/ privacy code scans.
Shifting Privacy Left Media
Where privacy engineers gather, share, & learn
Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.
Copyright © 2022 - 2024 Principled LLC. All rights reserved.
