Maria Lomeli

About me

I am a research engineer at Fundamental AI Research, Meta. Previously, I was a research scientist at Babylon Health, UK. Before that, I was a research associate, working with Zoubin Ghahramani at the Machine Learning group, CBL, University of Cambridge and member of Trinity Hall college. I studied my PhD at the Gatsby Unit, UCL, my supervisor was Yee Whye Teh.

Journal publications

  • Petroni, F., Broscheit, S., Piktus, A., Lewis, P., Izacard, G., Hosseini, L., Dwivedi-Yu, J., Lomeli, M., Schick, T., Mazaré, P.E., Joulin, A., Grave, E., Riedel, S., ''Improving wikipedia verifiability with AI'' Nature Machine Intelligence, 2023, Vol 5, pp 1142–1148, NMI.
  • Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., Grave, E., ''Atlas: few-shot learning with retrieval-augmented language models'' Journal of Machine Learning Research, 2023, Vol 24, pp 1-43, JMLR.
  • Mialon, G., Dessi, R., Lomeli, M., Nalmpantis, C., Pasunuru, R., Raileanu, R., Rosiere, B., Schick, T., Dwivedi-Yu, J., Celikyilmaz, A., LeCun, Y., Scialom, T., ''Augmented language models: a survey'', Transactions of Machine Learning Research, 2023, Vol 6, TLMR .
  • Valera, I., Pradier, M., Lomeli, M. and Ghahramani, Z., ''General Latent Feature Model for Heterogeneous Datasets'', Journal of Machine Learning Research, 2020, Vol 21 JMLR.
  • Lomeli, M., Rowland, M., Gretton, A. and Ghahramani, Z., ''Antithetic and Monte Carlo kernel estimators for partial rankings'', Statistics and Computing, 2019, Vol 29,1127–1147, StCo.
  • Lomeli, M., Favaro, S., Teh, Y. W.,'' A marginal sampler for -Stable Poisson-Kingman mixture models'', Journal of Computational and Graphical Statistics, 2017, Vol 26,pp 44-53 JCGS.
  • Favaro, S., Lomeli, M., Nipoti, B., Teh, Y.W., ''Stick-breaking representations of -stable Poisson-Kingman models'' , Electronic Journal of Statistics, 2014, Vol. 8, pp 1063-1085 EJS.
  • Favaro, S., Lomeli, M., Teh, Y.W.,''On a class of -stable Poisson-Kingman models and an effective marginalized sampler'', Statistics and Computing, 2014, Vol 25, pp 67-78 StCo.

    Proceedings

  • Cabannes, L., Beck, M., Szilvasy, G., Douze, M., Lomeli, M., Copet, J., Mazare, P., Synnaeve, G., and Jegou, H., 2026 ''Short window attention enables long-term memorization'' ICLR
  • Mekala, D., Weston, J., Lanchantin, J., Raileanu, R., Lomeli, M., Shang, J., Dwivedi-Yu, J., 2024, ''Toolverifier: Generalization to New Tools via Self-Verification '' EMNLP findings
  • Dwivedi-Yu, J., Schick, T., Jiang, Z., Lomeli, M., Lewis, P., Izacard, G., Grave, E., Riedel, S., Petroni, F., 2024, ''EditEval: an instruction-based benchmark for text improvements'' CoNLL
  • Lin, V. X., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M., Zettlemoyer, L., Yih, S., 2024. ''RA-DIT: Retrieval-Augmented Dual Instruction Tuning'' ICLR
  • Shi, W., Min, S., Lomeli, M., Chou, Z., Li, M., Lin, V., Smith, N. A., Zettlemoyer, L., Yih, S., Lewis, M. 2024, ''In-Context Pretraining: Language Modeling Beyond Document Boundaries'' ICLR (accepted as spotlight)
  • Schick, T., Dwivedi-Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., Scialom, T., 2023, ''Toolformer: language models can teach themselves to use tools'' Neural Information Processing Systems NeurIPS (accepted as an oral presentation)
  • Harman, M., Ahlgren, J., Berezin, M., Dulskyte, E., Dvortsova, I., George, J., Gucevska, N., Meijer, E., Spahr-Summers, J., Bojarczuk, K., Sapora, S. and Lomeli, M., 2021, ''Testing Web Enabled Simulation at Scale Using Metamorphic Testing'', International Conference on Software Engineering ICSE
  • Lomeli, M., Favaro, S.,Teh, Y.W., 2015, ''A hybrid sampler for Poisson-Kingman mixture models'', Neural information Processing Systems NeurIPS
  • Sejdinovic, D., Strathmann, H., Lomeli Garcia, M., Andrieu, C., Gretton, A., 2014,''Kernel Adaptive Metropolis-Hastings'', International Conference in Machine Learning ICML

    Workshops

  • Lupidi, A., Gemmell, C., Cancedda, N., Dwivedi-Yu, J., Weston, J., Foerster, J., Raileanu,R. and Lomeli, M.,2024, ''Source2Synth: synthetic data generation and curation grounded in real data sources'' arXiv (best paper award SynthData workshop, ICLR2025)
  • Gautam, D., Lomeli, M., Gourgoulias, K., Thompson, D., Johri, S., 2019, ''Masking schemes for universal marginalisers'', Advances in Approximate Bayesian Inference symposium
  • Bloem-Reddy, B., Mathieu, E., Foster, A., Rainforth, T., Ge, H., Lomeli, M., Ghahramani, Z., Teh, Y.W., 2017, ''Sampling and inference for discrete random probability measures in probabilistic programs'', Advances in Approximate Bayesian Inference workshop, NeurIPS

    Thesis

  • General Bayesian inference schemes in infinite mixture models
    PhD thesis, University College London
    UCL repository:     Doctoral dissertation link

    Preprints

  • Lomeli, M., Douze, M., Szilvasy, G., Cabannes, L., Copet, J., Sukhbaatar, S., Weston, J., Synnaeve, G., Mazare, P. and Jegou, H., 2025 ''Stochastic activations'' arXiv
  • Mazare, P., Szilvasy, G., Lomeli, M., Massa, F., Murray, N., Jegou, H., and Douze, M., 2025, ''Inference-time sparse attention with asymmetric indexing'' arXiv
  • Singh, A.K., Kocyigit, M. Y., Poulton, A., Esiobu, D., Lomeli, M., Szilvasy, G. and Hupkes, D., 2024, ''Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?' arXiv
  • Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazare, P.E., Lomeli, M., Hosseini, L. and Jegou, H. 2024, ''The faiss library'' arXiv