Stanford Machine Learning System Design

See past course for the last year's lectures. An ml system is designed iteratively. A generic system is typically made up of 4 components of the design process: 1) the project setup 2) data pipeline 3) modeling 4) serving. In early 2019, i started talking with stanford’s cs department about the possibility of coming back to teach. After almost two years in development, the course has finally taken shape. I’m excited to let you know that i’ll be teaching cs 329s: Machine learning systems design at stanford in january 2021.

A generic system is typically made up of 4 components of the design process: 1) the project setup 2) data pipeline 3) modeling 4) serving. In early 2019, i started talking with stanford’s cs department about the possibility of coming back to teach. After almost two years in development, the course has finally taken shape. I’m excited to let you know that i’ll be teaching cs 329s: Machine learning systems design at stanford in january 2021. The course wouldn’t have been. Gates computer science building 353 jane stanford way stanford, ca 94305. Emma brunskill recieves university of washington cs distinguished alumni award; Honorable mention for the 2021 acm doctoral dissertation award to dimitris tsipras

An ml system is designed iteratively. A generic system is typically made up of 4 components of the design process: 1) the project setup 2) data pipeline 3) modeling 4) serving. In early 2019, i started talking with stanford’s cs department about the possibility of coming back to teach. After almost two years in development, the course has finally taken shape. I’m excited to let you know that i’ll be teaching cs 329s: Machine learning systems design at stanford in january 2021. The course wouldn’t have been. Gates computer science building 353 jane stanford way stanford, ca 94305.

Popular Post

Image Gallery For Stanford Machine Learning System Design

Fundamentals of Machine Learning for Healthcare | Stanford Online


Fundamentals of Machine Learning for Healthcare | Stanford Online

Machine Learning fra Stanford


Machine Learning fra Stanford

Stanford Machine Learning Reviews & Compare


Stanford Machine Learning Reviews & Compare

Machine Learning | Stanford Online


Machine Learning | Stanford Online

Learn Machine Learning from Stanford University | Data science, Machine


Learn Machine Learning from Stanford University | Data science, Machine

Machine Learning System Design Stanford - MACHQI


Machine Learning System Design Stanford - MACHQI

Machine Learning Theory Stanford - machinei


Machine Learning Theory Stanford - machinei

Machine Learning | Stanford Online


Machine Learning | Stanford Online

Machine Learning System Design Stanford - MACHQI


Machine Learning System Design Stanford - MACHQI

Machine Learning with Graphs | Stanford Online


Machine Learning with Graphs | Stanford Online

Introduction to Machine Learning | Stanford Online


Introduction to Machine Learning | Stanford Online

Stanford University Machine Learning Certification - [PDF Document]


Stanford University Machine Learning Certification - [PDF Document]

Machine Learning Stanford Free - INEMACH


Machine Learning Stanford Free - INEMACH

Machine Learning System Design Stanford - MACHQI


Machine Learning System Design Stanford - MACHQI

Machine Learning Stanford Free - INEMACH


Machine Learning Stanford Free - INEMACH

Machine Learning Theory Stanford - machinei


Machine Learning Theory Stanford - machinei

Machine Learning System Design Stanford - MACHQI


Machine Learning System Design Stanford - MACHQI

Machine Learning Stanford Free - INEMACH


Machine Learning Stanford Free - INEMACH

Stanford Machine Learning Reviews & Compare


Stanford Machine Learning Reviews & Compare

Machine Learning Stanford Coursera Week 2 Quiz - INEMACH


Machine Learning Stanford Coursera Week 2 Quiz - INEMACH

Video Gallery For Stanford Machine Learning System Design

Stanford MLSys Seminar Episode 5: Chip Huyen



Episode 5 of the Stanford MLSys Seminar Series!

Principles of Good Machine Learning Systems Design
Speaker: Chip Huyen

Abstract:
This talk covers what it means to operationalize ML models. It starts by analyzing the difference between ML in research vs. in production, ML systems vs. traditional software, as well as myths about ML production.

It then goes over the principles of good ML systems design and introduces an iterative framework for ML systems design, from scoping the project, data management, model development, deployment, maintenance, to business analysis. It covers the differences between DataOps, ML Engineering, MLOps, and data science, and where each fits into the framework. It also discusses the main skills each stage requires, which can help companies in structuring their teams.

The talk ends with a survey of the ML production ecosystem, the economics of open source, and open-core businesses.

Speaker bio:
Chip Huyen is an engineer who develops tools and best practices for machine learning production. She’s currently with Snorkel AI and she’ll be teaching Machine Learning Systems Design at Stanford from January 2021. Previously, she was with Netflix, NVIDIA, Primer. She’s also the author of four bestselling Vietnamese books.

--

Check out our website for the schedule: mlsys.stanford.edu
Join our mailing list to get weekly updates: groups.google.com/forum/#!forum/stanford-mlsys-seminars/join

#machinelearning #ai #artificialintelligence #systems #mlsys #computerscience #stanford #stanforduniversity #snorkel #snorkelai #netflixai #nvidia

Stanford MLSys Seminar Episode 0: ML + Systems



Introductory video to the Stanford MLSys Seminar Series. Join us every week Thursday at 1:30-2:30 PM PT!

Check out our website for our schedule and more details: mlsys.stanford.edu/
Subscribe and join our mailing list for weekly updates! groups.google.com/forum/#!forum/stanford-mlsys-seminars/join

Machine Learning System Design | ML-005 Lecture 11 | Stanford University | Andrew Ng



Contents:

Prioritizing what to work on,
Error Analysis,
Error Metrics for Skewed Classes,
Trading Off Precision and Recall,
Data for Machine Learning,

Principles of Good Machine Learning Systems Design



This talk covers what it means to operationalize ML models. It starts by analyzing the difference between ML in research vs. in production, ML systems vs. traditional software, as well as myths about ML production.

It then goes over the principles of good ML systems design and introduces an iterative framework for ML systems design, from scoping the project, data management, model development, deployment, maintenance, to business analysis. It covers the differences between DataOps, ML Engineering, MLOps, and data science, and where each fits into the framework.

The talk ends with a survey of the ML production ecosystem, the economics of open source, and open-core businesses.

About:
Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: databricks.com/product/unified-data-analytics-platform

See all the previous Summit sessions: databricks.com/sparkaisummit/north-america/sessions

Connect with us:
Website: databricks.com
Facebook: facebook.com/databricksinc
Twitter: twitter.com/databricks
LinkedIn: linkedin.com/company/databricks/
Instagram: instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. databricks.com/databricks-named-leader-by-gartner

Deep Recommender Systems at Facebook feat. Carole-Jean Wu | Stanford MLSys Seminar Episode 24



Episode 24 of the Stanford MLSys Seminar Series!

Designing AI systems for deep learning recommendation and beyond
Speaker: Carole-Jean Wu

Abstract:
The past decade has witnessed a 300,000 times increase in the amount of compute for AI. The latest natural language processing model is fueled with over trillion parameters while the memory need of neural recommendation and ranking models has grown from hundreds of gigabyte to the terabyte scale. This talk introduces the underinvested deep learning personalization and recommendation systems in the overall research community. The training of state-of-the-art industry-scale personalization and recommendation models consumes the highest number of compute cycles among all deep learning use cases at Facebook. For AI inference, recommendation use cases consume even higher compute cycles of 80%. What are the key system challenges faced by industry-scale neural personalization and recommendation models? This talk will highlight recent advances on AI system development for deep learning recommendation and the implications on infrastructure optimization opportunities across the machine learning system stack. System research for deep learning recommendation and AI at large is at a nascent stage. This talk will conclude with research directions for building and designing responsible AI systems – that is fair, efficient, and environmentally sustainable.

Speaker bio:
Carole-Jean Wu is a Technical Lead and Manager at Facebook AI Research – SysML. Her work is in the domain of computer system architecture with particular emphasis on energy- and memory-efficient systems. Her research has pivoted into designing systems for machine learning execution at-scale, such as for personalized recommender systems and mobile deployment. In general, she is interested in tackling system challenges to enable efficient, responsible AI execution. Carole-Jean chairs the MLPerf Recommendation Benchmark Advisory Board, co-chaired MLPerf Inference, and serves on the MLCommons Board as a director. Carole-Jean received her M.A. and Ph.D. from Princeton and B.Sc. from Cornell. She is the recipient of the NSF CAREER Award, Facebook AI Infrastructure Mentorship Award, the IEEE Young Engineer of the Year Award, the Science Foundation Arizona Bisgrove Early Career Scholarship, and the Intel PhD Fellowship, among a number of Best Paper awards.

--
0:00 Starting Soon
4:46 Presentation
42:05 Discussion

The Stanford MLSys Seminar is hosted by Dan Fu, Karan Goel, Fiodar Kazhamiaka, and Piero Molino, Chris Ré, and Matei Zaharia.

Twitter:
twitter.com/realDanFu​
twitter.com/krandiash​
twitter.com/w4nderlus7

--

Check out our website for the schedule: mlsys.stanford.edu
Join our mailing list to get weekly updates: groups.google.com/forum/#!forum/stanford-mlsys-seminars/join

#machinelearning #ai #artificialintelligence #systems #mlsys #computerscience #stanford #fair #facebookai #recommendersystems #deeplearning

Google Machine Learning System Design Mock Interview



Today I am interviewing Dan for a second time on a machine learning system design problem centered around Youtube recommendations.

Want to be featured in the next mock interview video? Apply here: airtable.com/shrdQrwKK7xxGLm6l

👉 Subscribe to my data science channel: bit.ly/2xYkyUM
🔥 Get 10% off machine learning interview prep: interviewquery.com/?ref=datasciencejay

❓ Check out our ML interview course: interviewquery.com/courses/data-science-course/lessons/modeling
🔑 Get professional coaching here: interviewquery.com/coaching
🐦 Follow us on Twitter: twitter.com/interview_query

We go over how to build a recommendation system for Youtube, what kind of machine learning algorithms we would use, what the edge cases would be, how we could account for performance and scaling, and much more! For the full video please check out Interview Query

Quick Links:
0:55 - Youtube Recommendations Interview Question
3:35 - Collaborative Filtering
10:50 - Cold Start Problem?

More from Jay:
Read my personal blog: datastream.substack.com/
Follow me on Linkedin: linkedin.com/in/jay-feng-ab66b049/
Find me on Twitter: twitter.com/datasciencejay

Related Links:
Google ML Engineer Interview Profile: interviewquery.com/interview-experiences/google/machine-learning-engineer
Google data scientist interview: interviewquery.com/blog-the-google-data-scientist-interview

Stanford CS229: Machine Learning Course, Lecture 1 - Andrew Ng (Autumn 2018)



For more information about Stanford’s Artificial Intelligence professional and graduate programs, visit: stanford.io/2Ze53pq

Listen to the first lecture in Andrew Ng's machine learning course. This course provides a broad introduction to machine learning and statistical pattern recognition. Learn about both supervised and unsupervised learning as well as learning theory, reinforcement learning and control. Explore recent applications of machine learning and design and develop algorithms for machines.

Andrew Ng is an Adjunct Professor of Computer Science at Stanford University. View more about Andrew on his website: andrewng.org/

To follow along with the course schedule and syllabus, visit:
cs229.stanford.edu/syllabus-autumn2018.html

05:21 Teaching team introductions
06:42 Goals for the course and the state of machine learning across research and industry
10:09 Prerequisites for the course
11:53 Homework, and a note about the Stanford honor code
16:57 Overview of the class project
25:57 Questions

Machine Learning System Design Interview: YouTube Recommendations



Today I’m joined by Sachin, a senior data scientist. We’ll go over a machine learning system design question on how to build YouTube’s homepage recommendations system and optimize it.

Watch the rest on the question page: interviewquery.com/questions/youtube-recommendations

Previous YouTube Recommendations interview with Dan: youtu.be/9TxN1VoPYq4

Follow Sachin on LinkedIn: linkedin.com/in/venugopal-mani-umn/

His sources:
HRNN Paper: openreview.net/pdf?id=8QFKbygVy4r
Prof Weinberger's channel: youtube.com/channel/UC7p_I0qxYZP94vhesuLAWNA/featured
Prof. Justin Johnson's course: youtube.com/watch?v=dJYGatp4SvA&list=PL5-TkQAfAZFbzxjBHtzdVCWE0Zbhomg7r

Want to be featured in the next mock interview video? Apply here: airtable.com/shrdQrwKK7xxGLm6l

👉 Subscribe to my data science channel: bit.ly/2xYkyUM
🔥 Get 10% off machine learning interview prep: interviewquery.com/?ref=datasciencejay

❓ Check out our ML interview course: interviewquery.com/courses/data-science-course/lessons/modeling
🔑 Get professional coaching for your next interview: interviewquery.com/coaching
🐦 Follow us on Twitter: twitter.com/interview_query

00:00 - Question
00:17 - Clarifying Questions
03:57 - System Architecture of an ideal recommender
13:24 - Various Components of the architecture
26:25 - Online/Offline evaluation of the recommendation
30:03 - AB Test Launch

More from Jay:
Read my personal blog: datastream.substack.com/
Follow me on Linkedin: linkedin.com/in/jay-feng-ab66b049/
Find me on Twitter: twitter.com/datasciencejay

#DataScience #MachineLearning #SystemDesign #DataScientist #DataEngineer #MachineLearningEngineer

Machine Learning System Design (YouTube Recommendation System)



As an excellent Machine Learning System Design example, I am going through the following paper:

"Recommending What Video to Watch Next: A Multitask Ranking System" by Google Inc. presented at RecSys 2019.

[PDF] daiwk.github.io/assets/youtube-multitask.pdf

// MY RECOMMENDATIONS:
Great "Machine Learning" Book:
📚 Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: amzn.to/3pwbjjE

Great "System Design" Book:
📚 Designing Data-Intensive Applications: amzn.to/3n05KYW

#machinelearning #systemdesign #recommendationsystem #recommendersystem #recommendation #rankingsystem #ranking #google #youtube #ai #ml #artificialintelligence #mlsystemdesign

Lecture 11.1 — Machine Learning System Design | Prioritizing What To Work On — [ Andrew Ng]



Hey guys! In this channel, you will find contents of all areas related to Artificial Intelligence (AI). Please make sure to smash the LIKE button and SUBSCRIBE to our channel to learn more about these trending topics, and don’t forget to TURN ON your YouTube notifications!

Thanks & Happy Learning 🙂
.

Designing Machine Learning Systems | book summary | Read a book with me



Hi, in this video, I am going to summarize the book Designing Machine Learning Systems by Chip Huyen. This book covers a lot of machine learning system-related information including data formats, how to deal with imbalanced data, how to deal with missing data, how to evaluate models, how to improve model performance, how to monitor and test models in production, how to deal with data shifts, how to do continual learning, and many more. I’m going to summarize and highlight key points of the book, especially the following book chapters:
- Data Engineering Fundamentals
- How to create training data
- Feature Engineering
- Model Development and Offline Evaluation
- Model Deployment and Prediction Service
- Data Distribution Shifts and Monitoring
- Continual Learning and Test in Production
Hope you find this video helpful. Thanks for watching 🤗

📚 Book link 📚
- amzn.to/3mRH7QD

✨ Mentioned in the video✨
- My video on Thompson Sampling: youtube.com/watch?v=TdjOAfk7iVA
- My video on model behavioral testing: youtube.com/watch?v=MecK4Bbw3Fs
- My blog post on PyScript: anaconda.cloud/pyscript-python-in-the-browser

🛠 My gear 🛠
- Tablet (newer version): amzn.to/3bMQ0ZB
- Computer: amzn.to/39sTujd
- Camera: amzn.to/3xx1QOH
- Hue lights: amzn.to/3ba3H4E

🌼 About me 🌼
Sophia Yang is a Senior Data Scientist working at a tech company.

🔔 SUBSCRIBE to my channel: youtube.com/c/SophiaYangDS?sub_confirmation=1

⭐ Stay in touch ⭐
📚 DS/ML Book Club: discord.com/invite/6BremEf9db
▶ YouTube: youtube.com/SophiaYangDS
✍️ Medium: sophiamyang.medium.com
🐦 Twitter: twitter.com/sophiamyang
🤝 Linkedin: linkedin.com/in/sophiamyang/
📧 Email: [email protected]
💚 #datascience #machinelearning

Machine Learning System Design Interview - Valerii Babushkin



Links:

- Valerii's telegram channel (in Russian): t.me/cryptovalerii

Join DataTalks.Club: datatalks.club/slack.html
Our events: datatalks.club/events.html

AWS re:Invent 2020: Designing better ML systems: Learnings from Netflix



Data science at Netflix goes far beyond eponymous recommendation systems and touches every aspect of the business, from optimizing content delivery to fighting fraud. Netflix’s unique culture affords its data scientists extraordinary freedom of choice in tools, which results in an ever-expanding set of machine learning (ML) approaches and systems. In 2019, Netflix open-sourced Metaflow, its human-centric ML platform. In this session, Netflix shares some lessons learned in its multi-year journey building the ML systems that Metaflow incorporates, covering a diverse range of scale from one-time experimentation on laptops to large-scale model training and serving systems on AWS.

Learn more about re:Invent 2020 at bit.ly/3c4NSdY

Subscribe:
More AWS videos bit.ly/2O3zS75
More AWS events videos bit.ly/316g9t4

#AWS #AWSEvents

ML Ops System Design feat Comet ML CEO Gideon Mendels | Stanford MLSys #44



Episode 44 of the Stanford MLSys Seminar Series!

MLOps System Design for Development and Production
Speaker: Gideon Mendels

Abstract:
While ML model development is a challenging process, the management of these models becomes even more complex once they're in production. Shifting data distributions, upstream pipeline failures, and model predictions impacting the very dataset they’re trained on can create thorny feedback loops between development and production. In this talk, we’ll examine some naive ML workflows that don’t take the development-production feedback loop into account and explore why they break down. Next, we'll showcase some system design principles that will help manage these feedback loops more effectively. Finally, we’ll examine several industry case studies where teams have applied these principles to their production ML systems.

--

Stanford MLSys Seminar hosts: Dan Fu, Karan Goel, Fiodar Kazhamiaka, and Piero Molino
Executive Producers: Matei Zaharia, Chris Ré

Twitter:
twitter.com/realDanFu​
twitter.com/krandiash​
twitter.com/w4nderlus7

Intro music:
Heading Home by Nekzlo @nekzlo
Music provided by Free Music for Vlogs youtu.be/YGL35p7LLqo

--
0:00 Starting soon
10:21 Presentation
34:24 Discussion

Check out our website for the schedule: mlsys.stanford.edu
Join our mailing list to get weekly updates: groups.google.com/forum/#!forum/stanford-mlsys-seminars/join

#machinelearning #ai #artificialintelligence #systems #mlsys #computerscience #stanford #cmu #optimizingml #optimization #automaticoptimization

Lecture 11.5 — Machine Learning System Design | Data For Machine Learning — [Andrew Ng]



Hey guys! In this channel, you will find contents of all areas related to Artificial Intelligence (AI). Please make sure to smash the LIKE button and SUBSCRIBE to our channel to learn more about these trending topics, and don’t forget to TURN ON your YouTube notifications!

Thanks & Happy Learning 🙂
.

ML System Design: Feature Store



AppliedAICourse.com
Additional References:
medium.com/data-for-ai/comprehensive-and-comparative-list-of-feature-store-architectures-for-data-scientists-and-big-data-86ea8c4d853b

Stanford MLSys Seminar Episode 2: Matei Zaharia



Episode 2 of the Stanford MLSys Seminar Series!

Machine Learning at Industrial Scale: Lessons from the MLflow Project
Speaker: Matei Zaharia

Abstract:
Although enterprise adoption of machine learning is still early on, many enterprises in all industries already have hundreds of internal ML applications. ML powers business processes with an impact of hundreds of millions of dollars in industrial IoT, finance, healthcare and retail. Building and operating these applications reliably requires infrastructure that is different from traditional software development, which has led to significant investment in the construction of “ML platforms” specifically designed to run ML applications. In this talk, I’ll discuss some of the common challenges in productionizing ML applications based on experience building MLflow, an open source ML platform started at Databricks. MLflow is now the most widely used open source project in this area, with over 2 million downloads a month and integrations with dozens of other products. I’ll also highlight some interesting problems users face that are not covered deeply in current ML systems research, such as the need for “hands-free” ML that can train thousands of independent models without direct tuning from the ML developer for regulatory reasons, and the impact of privacy and interpretability regulations on ML. All my examples will be based on experience at large Databricks / MLflow customers.

Speaker bio:
Matei Zaharia is an Assistant Professor of Computer Science at Stanford University and Chief Technologist at Databricks. He started the Apache Spark project during his PhD at UC Berkeley in 2009, and has worked broadly on other cluster computing and analytics software, including MLflow and Delta Lake. At Stanford, Matei is a co-PI of the DAWN Lab doing research on infrastructure for machine learning. Matei’s work was recognized through the 2014 ACM Doctoral Dissertation Award, an NSF CAREER Award, and the US Presidential Early Career Award for Scientists and Engineers (PECASE).

--

Check out our website for the schedule: mlsys.stanford.edu
Join our mailing list to get weekly updates: groups.google.com/forum/#!forum/stanford-mlsys-seminars/join

Machine Learning Course - 23. ML Design Pattern - Ranking



A full university-level machine learning course - for free. New lectures every week.

Designed as a first course for engineers, program managers, and data professionals who want to learn: the details of important machine learning algorithms; professional model building; and design patterns for building practical machine learning systems.

This lecture covers: Machine learning case study, ranking machine learning, mean average precision, ranking algorithm, ranknet.

And you can view the entire course in this playlist: youtube.com/playlist?list=PLrQmbzbRJ5mwDinvDEJ5B-KDZlPM-sCYO

Learn more (and find slides for this lecture) at: livingmachinelearning.com/course.html

LinkedIn Machine Learning Mock Interview - Design a recommendation engine



Today Ved takes us through how to answer a machine learning mock interview question from our weekly challenge question at Interview Query. This question was asked by a job search company like LinkedIn, Indeed, or Ziprecruiter.

Follow Ved on LinkedIn: linkedin.com/in/ved-piyush/

Want to be featured in the next mock interview video? Apply here: airtable.com/shrdQrwKK7xxGLm6l

👉 Subscribe to my data science channel: bit.ly/2xYkyUM
🔥 Get 10% off machine learning interview prep: interviewquery.com/?ref=datasciencejay

❓ Check out our ML interview course: interviewquery.com/courses/data-science-course
🔑 Get professional coaching here: interviewquery.com/coaching
🐦 Follow us on Twitter: twitter.com/interview_query

Quick Links
Ved's background and introduction: 0:00
Interview question: 2:20
Ved whiteboarding: 5:10
Practical application: 15:00
Deployment: 25:00

More from Jay:
Read my personal blog: datastream.substack.com/
Follow me on Linkedin: linkedin.com/in/jay-feng-ab66b049/
Find me on Twitter: twitter.com/datasciencejay

Trending Search

All in One Design Ideas - Design Ideas

Stanford Machine Learning System Design, Stanford MLSys Seminar Episode 5: Chip Huyen, 9.25 MB, 06:44, 39,045, Stanford MLSys Seminars, 2020-11-13T00:11:27.000000Z, 19, Fundamentals of Machine Learning for Healthcare | Stanford Online, online.stanford.edu, 775 x 436, jpeg, I wanted to ask you guys whether it is normal to get overwhelmed while learning machine learning. I have been studying since almost last 6 months and i feel like i am getting nowhere. The lecture slides, notes, tutorials, and assignments will be posted online here as the course progresses. All deadlines are at 11:59pm pst. , 20, stanford-machine-learning-system-design, Design Ideas

© Copyright 2022. All Rights Reserved. Design Ideas