Thought-provocative new paper from @geoffreyhinton: what if we could replace backpropagation with something better?
Martin Görner
1,291 posts
Martin Görner
@martin_gorner
AI/ML engineer. Previously at Google: Product Manager for Keras and TensorFlow and developer advocate on TPUs. Passionate about democratizing Machine Learning.
France
Joined May 2016
- This is sweet 🥧 ! arxiv.org/abs/2202.01197 Finally a solid way of of teaching a neural network to know what it does not know. (OOD = Out Of Domain, i.e. not one of the classes in the training data.) Congrats @SharonYixuanLin @xuefeng_du @MuCai7
00:00- Self-supervised learning techniques on images are making steady progress. Here are Masked Auto Encoders: arxiv.org/abs/2111.06377 I's impressive how much the neural network can reconstruct.
- The classic "Tensorflow without a PhD" tutorial has been updated for Keras and Tensorflow 2.2: codelabs.developers.google.com/codelabs/cloud… After many years, still a good place to start learning about neural networks, relu, softmax, cross-entropy, learning rates, dropout, batch norm, ... Enjoy!
- I believe a dev can get up to speed on neural networks in 3h and then learn by himself. Ready for a crash course? youtu.be/qyvlt7kiQoI /1
- A fascinating overview of research into "Dataset Distillation" arxiv.org/abs/2301.07014 How to train a neural network on fewer data points and achieve the same performance as when training on the original dataset.
- Large Language Models are getting good at formal logic: arxiv.org/abs/2212.13894 LAMBADA: Backward Chaining for Automated Reasoning.
- Here is an end-to-end canonical sample for training a model on Cloud TPUs in Keras. It has full code for loading the data from scratch using tf.data.Dataset and also exporting the trained model to ML Engine for inference. Colab notebook: colab.research.google.com/github/GoogleC…
- The book is out🎉😄🎉. Official announcement from O'Reilly.[NEW RELEASE] Practical Machine Learning for Computer Vision -- Learn how to design, train, evaluate, and predict with models written in TensorFlow or Keras, w/ experts @lak_gcp @martin_gorner & Ryan Gillard oreil.ly/4ugsQ #OReillyExperts #NewBook
00:00 - This looks like the Vision Transformers architecture we have been waiting for: MaxViT arxiv.org/abs/2204.01697 1/ State of the Art accuracy on ImageNet (no pre-training on huge datasets) 2/ Linear complexity wrt. image size (thanks to a clever attention design)
- Replying to @martin_gorner and @geoffreyhintonAnd finally, Hinton points out that FF is easily implementable with very power-efficient analog circuits. This could allow us to run very large language models in a fraction of a Watt! Read the paper for more Hinton's thoughts I have omitted. cs.toronto.edu/~hinton/FFA13.…
- TPUs in Google Colaboratory, now with less boilerplate code. See the Keras TPU sample here: colab.research.google.com/github/GoogleC… google.colab.auth.authenticate_user() to propagate your credentials to the backend and the TPU. TPUClusterResolver() to find your TPU. That's it!
