Pinned
70 lines of code to massively accelerate LLM training!
arxiv.org/abs/2409.03137
Pierre Ablin
899 posts
Pierre Ablin
@PierreAblin
- What better way to start the year than with a whiteboard with a view ?
- 🍏🍏🍏 Apple ML Research in Paris has multiple open research internship positions! 🍎🍎🍎 We are looking for Ph.D. students interested in generative modeling, optimization or uncertainty quantification, with applications to challenging scientific problems. Details below 👇👇👇
- Roger Penrose just won a Nobel prize in physics for black holes theory ! He is also famous for the Moore-Penrose inverse, which extends the notion of matrix inverse to rectangular and singular matrices. en.m.wikipedia.org/wiki/Moore%E2%…
- 🍏 Apple ML research in Paris has multiple open internship positions!🍎 We are looking for Ph.D. students interested in generative modeling, optimization, large-scale learning or uncertainty quantification, with applications to challenging scientific problems. Details below 👇
- New paper out : Momentum Residual Neural Networks ! Introducing a new drop-in replacement for any ResNet that makes it invertible, thus saving loads of memory ! With @m_e_sander @mblondel_ml & @gabrielpeyre Preprint : arxiv.org/abs/2102.07870 Accepted at ICML 🍾🍾🍾 1/6
- Apple ML Research in Paris has open research internship positions ! Looking for PhD students with background in ML / optimization. Internships are onsite, should happen anytime from now until sep. 2023. Please DM me if interested :)
- An important intuition about Stochastic Gradient Descent (SGD) is that when you are far from the solutions, individual stochastic gradients will more or less point in the same direction, but at optimum, since their average is 0, they will all point in different directions !
GIF - New paper out : a simple way to have optimal transport with sparse displacements🎆 "Monge, Bregman and Occam: Interpretable Optimal Transport in High-Dimensions with Feature-Sparse Maps" arxiv.org/abs/2302.04065 w. @CuturiMarco and Michal Klein A small thread👇
GIF - Here are the first 100 principal components of the Imagenet dataset ! I had never seen this before, so I thought I'd give it a try :) Nothing too surprising, it looks like the principal components of most natural image datasets.
- 🍏🍏🍏 Come work with us at Apple Machine Learning Research! 🍏🍏🍏 Our team focuses on curiosity-based, open research. We work on several topics, including LLMs, optimization, optimal transport, uncertainty quantification, and generative modeling. Infos 👇
- Illustration of the Lasso and its path in 2D: for t small enough, the solution is sparse!
GIF - TIL you can see double descent when fitting 1d polynomials: - When degree < # samples: low variance, high bias - when degree ~ # samples: super high variance - when degree >> # samples: low norm solution, you get interpolation + extrapolation ! Nice ref: arxiv.org/abs/1903.09139
GIF