Apple and Meta have published a monstruously elegant compression method that encodes model weights using pseudo-random seeds.
The trick is to approximate model weights as the linear combination of a randomly generated matrix with fixed seed, and a smaller vector t.
Dante
1,084 posts
- Just read this paper from HKU, that presents a simple approach to get smaller transformer models to train as well as larger ones. Their key insight is to tradeoff a smaller parameter count for more computation at training time by "looping" over transformer blocks recursively.
- Replying to @seanxthielenApple should acquire Anthropic and then immediately replace Claude with Siri
- Hello ML friends. Over the past few months I've been working on EZKL in collaboration with @jasonmorton with support from @0xPARC. This library enables anyone to convert a pytorch or tensorflow computational graph into a ZK-SNARK. github:
- Some of y'all have been messaging us at 4am begging for tree based models in ezkl. well ... we HEARD U. you can now convert sklearn tree based models and XGBoost models into zk-circuits. time to start planting trees on-chain. writeup + jupyter nb: hackmd.io/oczJRjWDQQO4xv…
- We, @ezklxyz with MIT and Microsoft Research, have published a methodology to cryptographically prove that closed-source AI models achieve claimed performance metrics without revealing model weights.
- Zero knowledge proofs are going mainstream and being integrated into Google products
- Replying to @CamutoDanteSome of these ideas are pretty similar to some stuff @mjfw3 had kicked around during our PHDs: the idea that random projections could be sufficiently good approximations of an original trained matrix / a solution to an optimization problem. proceedings.mlr.press/v130/camuto21b…
- Zero-knowledge systems face a fundamental vulnerability: while they can prove computations were done correctly, they can't always verify if the input data itself is legitimate. Let's call this "the garbage in garbage out" problem.
- Replying to @CamutoDanteAll you need to store now are the seed and the smaller set t. So you effectively reduce memory consumption during inference for small increases in compute. Using LFSR registers this cost can be mitigated, as these registers are particularly effective for random num generation.
- Replying to @CamutoDanteBut here leveraging the quirks of LSFRs is straight up insane and means that they get very little increase in computational costs, lower memory requirements for running large models, all the while retaining most of the perf of the original model. arxiv: arxiv.org/pdf/2410.10714
- Great to see the ecosystem grow but bad benchmarking is kind of tired and sloppy. We went through your benchmarking code @lagrangedev and it looks like you’re artificially handicapping @ezklxyz proving times by fixing the number of rows in our circuits to a very large number.Introducing DeepProve—Lagrange’s zkML Library—a breakthrough in verifiable AI inference. We can now verify AI decisions instead of blindly trusting black-box models. And we can do it up to 158x faster than ever before. The future of AI is ZK. The future of humanity is Lagrange:
00:00











