# Data Engineering and MLOps Home # Books - [~] Chip Huyen's [Designing Machine Learning Systems](https://github.com/chiphuyen/dmls-book) - notes: [[Designing Machine Learning Systems]] - Chip Huyen's [AI Engineering](https://www.amazon.com/dp/1098166302?&linkCode=sl1&tag=chiphuyen-20&linkId=0a4e5ad4b14080d44c42640550a9291e&language=en_US&ref_=as_li_ss_tl) - Chip Huyen's [ML Interviews](https://huyenchip.com/ml-interviews-book/) # Courses - Andrew Ng [https://www.andrewng.org/courses/]() - [Generative AI for Everyone](https://www.deeplearning.ai/courses/generative-ai-for-everyone/) - [AI for Everyone](https://www.deeplearning.ai/courses/ai-for-everyone/?utm_medium=referral&utm_source=andrew-website) 6 hour non-technical course - [Machine Learning Engineering for Production](https://www.youtube.com/watch?v=NgWujOrCZFo&list=PLkDaE6sCZn6GMoA0wbpJLi3t34Gd8l0aK) - Chip Huyen - [ML Curriculum](https://huyenchip.com/2019/08/05/free-online-machine-learning-curriculum.html) - [Stanford CS329S Machine Learning Systems Design](https://stanford-cs329s.github.io/syllabus.html) - [MLOps Guide](https://huyenchip.com/mlops/) - [https://www.deeplearning.ai/]() # Tools - Data Processing - Python Dataframe Libraries: Pandas, Polars, DuckDB, Arrow - Distributed Systems: Spark, Ray - Query Engines: SparkSQL, Trino, Datafusion - Query Frontends: MLFlow, Superset - Machine Learning Libraries: Scikit-Learn, PyTorch, Tensorflow - Monitoring and Observability: MLFlow, Grafana, Prometheus - Orchestration: Apache Airflow, MLFlow - [Open Source LLM Tools](https://huyenchip.com/llama-police) # Articles - [gcp real terms for ai](https://www.youtube.com/playlist?list=PLIivdWyY5sqLvGdVLJZh2EMax97_T-OIB) # Papers 1. **[Transformer (deep learning architecture) – Wikipedia](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture))** A comprehensive overview of the transformer model, its architecture, and applications in deep learning. 2. **[An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/abs/2010.11929)** Introduces the Vision Transformer (ViT), showing that pure transformers can outperform CNNs on image classification tasks. 3. **[BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805)** Presents BERT, a transformer-based model pre-trained on masked language modeling and next sentence prediction. 4. **[Attention Is All You Need](https://arxiv.org/abs/1706.03762)** The seminal paper that introduced the Transformer architecture and the attention mechanism, replacing RNNs for many NLP tasks. 5. **[FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://arxiv.org/abs/2210.05189)** Proposes FlashAttention, a highly efficient attention algorithm that reduces memory and speeds up computation. 6. **[vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention](https://arxiv.org/abs/2308.16512)** Introduces vLLM, a high-throughput LLM serving engine using a novel attention pattern called PagedAttention. 7. **[neuralmagic/vllm-flash-attention – GitHub](https://github.com/neuralmagic/vllm-flash-attention)** Code and resources for efficient LLM serving using FlashAttention, developed by Neural Magic. 8. **[Accelerating Training of Transformer Models with Actively Compressed Activation Maps – CIDR 2021](https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf)** Discusses techniques to reduce memory and improve training performance of transformer models through activation compression. # Blogs - https://huyenchip.com/ - https://www.andrewng.org/