Pinned
For years since the GPT-2 paper, emergent in-context learning (ICL) from 'next-token' training has been treated as something deeply tied to 𝐡𝐮𝐦𝐚𝐧 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞. But … is it?
Thrilled to share our latest result: 𝗚𝗲𝗻𝗼𝗺𝗶𝗰🧬 𝗺𝗼𝗱𝗲𝗹𝘀 𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝙤𝙣𝙡𝙮 𝗼𝗻
Daniel Khashabi 🕊️
2,001 posts
Daniel Khashabi 🕊️
@DanielKhashabi
I play with intuitions and data.
Now: @jhuclsp @jhucompsci
Past: @allen_ai @uwnlp @Penn @cogcomp, @Illinois_Alma, @MSFTResearch
He/Him
Joined October 2012
- Life update: Thrilled to announce that I will join Johns Hopkins University @jhuclsp @jhucompsci @JohnsHopkins as an assistant professor of computer science in the fall! This is an honor of a lifetime and I will do my best to rise to the occasion.
- For my first course at @jhuclsp, I am leading a class on recent developments in "self-supervised models." Here is the list of the papers and slides we cover: self-supervised.cs.jhu.edu Would love to hear Twitter's suggestions for additional exciting developments to discuss!🤗
- For anyone interested, we've updated this slide deck on instruction-tuning/RLHF for my class! - PDF: self-supervised.cs.jhu.edu/sp2023/files/2… - PPT: self-supervised.cs.jhu.edu/sp2023/files/2… Would love to hear your feedback!Since prompting, instruction tuning, RLHF, ChatGPT etc are such new and fast-moving topics, I haven't seen many university course lectures covering this content. So we made some new slides for this year's CS224n: NLP w/ Deep Learning course at @Stanford! web.stanford.edu/class/cs224n/s…
- Today we are releasing GENIE🧞, a human-in-loop leaderboard for the evaluation of text generation tasks! We view this as a step forward towards streamlining human evaluation and making it more accessible. genie.apps.allenai.org arxiv.org/abs/2101.06561 #NLP
- It is concerning that an increasing number of research papers base the core of their studies/findings on the new GPT3 models (especially 'davinci-002'), which we know little about training/tuning. How can we do scientific research on these murky foundations?
- Self-supervised models are a must-know for CS undergrads entering the job market. This semester I taught my first undergrad/MS course on these models, exploring their impact.The course content (slides/assignments) is online for those interested: self-supervised.cs.jhu.edu/sp2023/
- Congrats to whoever graduating this year!! 👏 Please take a few minutes to read and share my piece for @dailypenn on why "I am mourning at graduation", thanks to politics: thedp.com/article/2019/0… @UndoFamilyBan #travelban
- Excited that our big collaborative effort, "ParsiNLU: A Suite of Language Understanding Challenges for Persian" will appear in TACL'21! If you're working on multilingual/cross-lingual NLP, give it a look! Paper:
- Overheard someone say GPT-4 is "the end of NLP and CV". That is as absurd as suggesting that iPhone's first release in 2007 marked the end of phone technology. This is not "an end" but rather the beginning of a new era of technological advancements and applications.
- 📢 GooAQ 🥑: 3 million questions/answers, with a variety of answer types! Draft: bit.ly/3gkeZDC Data: github.com/allenai/gooaq 🚨Spoiler alert:🚨 we observe that short- vs long-answer questions behave differently!
- 📢 New Preprint📢 Is it possible to provide meaningful textual interpretations of continuous prompts? Find out in our new preprint! arxiv.org/abs/2112.08348 Joint work with @sewon__min, @Lianhuiq @sameer_ @wellecks @HannaHajishirzi @tusharkhot @YejinChoinka and others.
- Excited to highlight our work, "Cross-Task Generalization via Language Instructions" arxiv.org/pdf/2104.08773… TLDR; Language instructions improve generalization to "unseen" tasks. The gains increase w/ more observed tasks. Joint w/ @Swarooprm7 @cbaral @HannaHajishirzi
- Excited about [re]joining Allen AI @allen_ai! Over the past few years, AI2 has been at the forefront of key developments in AI/NLP & it's an honor to be part of this vibrant community.
