Bayesiains Reading Group

Time and place: 2–3:30pm on roughly alternate Mondays, IF 1.16.

Rota

Topics/papers

Reset the list for 2020. Brainstorm of ideas, not a schedule:

Approximate Bayesian inference, calibration, ...

• ...

Transfer learning, self-supervision, and friends

• ...

Similarity and nearest neighbours

• HNSW for approximate nearest neighbours does well in some benchmarks. (Although FAISS offers some discussion of the choices for different regimes.)

• nmslib implements HNSW, and has a reading list.

• Any NN-graph methods? Classics like isomap and LLE. Discuss how nearest neighbours can fit into more recent methods?

Optimization

Anything that helps making practical choices, or gives insight into what choices matter?

Batch size:

• Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model

• Measuring the Effects of Data Parallelism on Neural Network Training

Learning rates:

• ...

Discrete models

(Not continuous-like data, such as quantized images.)

Fast generation of high-dim patterns?

• Non-Autoregressive Neural Machine Translation
• Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
• RBMs?

Methods for latent variable models. Concrete distribution, REBAR/RELAX...?

Unbiased estimation in learning

• More on exact estimation with MCMC?

• Unbiased scalable softmax optimization (although critical review

• Efficient Optimization of Loops and Limits with Randomized Telescoping Sums

• ...?

Reinforcement learning?

• ...?

Boosted trees

• A look at what's inside widely-used tools like XGBoost, LightGBM, or CatBoost?

Embedding different data types

• Graph Neural Networks?