Hiroki Furuta (@frt03

Hiroki Furuta

160 posts

Hiroki Furuta

@frt03_

Research Scientist at @GoogleDeepMind / PhD from The University of Tokyo @Matsuo_Lab / AI Agent / AI Alignment / LLMs / Deep RL

Joined August 2018

Hiroki Furuta
@frt03_
Sep 26, 2024
RLHFに関する論文が #NeurIPS 2024に採択されました！🎉 DPOが好みの割合を表すソフトな選好ラベルを活用できるように拡張し、オフラインとオンラインの両方でより良いLLMのアラインメントを達成します Google DeepMindでのインターン中の成果です
Hiroki Furuta
@frt03_
Sep 26, 2024
Our paper on RLHF: Geometric-Averaged Preference Optimization for Soft Preference Labels was accepted to #NeurIPS 2024! To deal with over-optimization in DPO, proportional soft labels taken from majority voting/AI feedback can adjust the gradient scale. arxiv.org/abs/2409.06691
42K
Hiroki Furuta
@frt03_
Jan 23, 2025
Our paper about the evaluation of sparse autoencoder (SAE) was accepted at #ICLR2025 🎉 We propose leveraging the representation similarity of polysemous words (e.g. "space") to measure the quality of SAE. This is work done at The Univerisity of Tokyo @Matsuo_Lab
28K
Hiroki Furuta
@frt03_
Jan 23, 2025
東大松尾・岩澤研究室で行った共著の論文が #ICLR 2025に採択されました🎉 GPT-4/LLaMA/Gemmaなどにも用いられている、LLMの内部回路の解釈性(mechanistic interpretability)を得るためのツールであるSAEの性能を、多義語を用いて測る新たな指標を提案しました
Hiroki Furuta
@frt03_
Jan 23, 2025
Our paper about the evaluation of sparse autoencoder (SAE) was accepted at #ICLR2025 🎉 We propose leveraging the representation similarity of polysemous words (e.g. "space") to measure the quality of SAE. This is work done at The Univerisity of Tokyo @Matsuo_Lab
17K
Hiroki Furuta
@frt03_
Sep 29, 2021
既存の深層強化学習のアルゴリズムを分類した論文がNeurIPS2021に採択されました！松尾研、@Tdash_Kozさん、 @shaneguMLさんの共同研究の成果です arxiv.org/abs/2103.17258
Hiroki Furuta
@frt03_
Jan 24, 2023
"A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation" was accepted at #ICLR2023 as notable-top-25% (w/ @yusuke_iwasawa_ @ymatsuo @shaneguML) We propose a data distillation pipeline for large-scale models in RL. arxiv.org/abs/2211.14296
46K
Hiroki Furuta
@frt03_
Feb 5, 2025
Our new preprint on inference-time alignment for diffusion models is out🎉 To improve text-to-video generation, we propose Diffusion Latent Beam Search with Lookahead Estimator. We should allocate the computation as: Lookahead Steps >> Search Budget >> Denoising Steps 1/8
GIF
18K
Hiroki Furuta
@frt03_
May 25, 2023
Yesterday I droped by AK meetup in Tokyo and have dinner with him! Really exciting night🤗 Enjoy your rest of stay in Japan @_akhaliq
32K
Hiroki Furuta
@frt03_
Dec 6, 2024
Google DeepMind Japanでのインターン中の成果をプレプリントで公開しました！物体がダイナミックに動くような場面の生成では不正確な出力が多いtext-to-videoの動画生成モデルを、GeminiなどマルチモーダルなVLMからのAI feedbackによるRL finetuningを通して改善します
Hiroki Furuta
@frt03_
Dec 5, 2024
Text-to-video models can generate photorealistic scenes but still struggle to accurately depict dynamic object interactions😢 Our new preprint addresses this through RL finetuning with AI feedback from VLMs capable of video understanding (e.g. Gemini, etc)🎉 1/7
GIF
13K
Hiroki Furuta
@frt03_
May 8, 2021
強化学習のベンチマーク環境の難易度を定量評価するための指標(PIC/POIC)を提案した論文がICML2021に採択されました！論文: arxiv.org/abs/2103.12726 解説資料(日本語):
DLHacks
@DL_Hacks
May 6, 2021
PIC/POICは報酬/最適性変数と方策のパラメータ間の相互情報量によって定義され、強化学習ベンチマーク環境の"解きやすさ"を特定のアルゴリズムに依らず定量評価できる。また学習せずにいくつかの実験パラメータが調整できる。 slideshare.net/DeepLearningJP…
arxiv.org
Policy Information Capacity: Information-Theoretic Measure for...
Progress in deep reinforcement learning (RL) research is largely enabled by benchmark task environments. However, analyzing the nature of those environments is often overlooked. In particular, we...
Hiroki Furuta
@frt03_
Aug 24, 2023
FORBES JAPAN 30 UNDER 30 2023に選んで頂きました。これからも良い研究ができるように頑張ります！ #u30fj
FORBES JAPAN 30 UNDER 30 2023｜日本発「世界を変える30歳未満」120人

From forbesjapan.com
8.9K
Hiroki Furuta
@frt03_
Sep 26, 2024
Our paper on RLHF: Geometric-Averaged Preference Optimization for Soft Preference Labels was accepted to #NeurIPS 2024! To deal with over-optimization in DPO, proportional soft labels taken from majority voting/AI feedback can adjust the gradient scale. arxiv.org/abs/2409.06691
35K
Hiroki Furuta
@frt03_
Dec 5, 2024
Text-to-video models can generate photorealistic scenes but still struggle to accurately depict dynamic object interactions😢 Our new preprint addresses this through RL finetuning with AI feedback from VLMs capable of video understanding (e.g. Gemini, etc)🎉 1/7
GIF
27K
Hiroki Furuta
@frt03_
Jan 16, 2024
Two papers on web agents are accepted at #ICLR2024 (w/ Oral)! Thank you to all the collaborators. [Poster] Multimodal Web Navigation arxiv.org/abs/2305.11854 [Oral] A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis arxiv.org/abs/2307.12856
arxiv.org
Multimodal Web Navigation with Instruction-Finetuned Foundation Models
The progress of autonomous web navigation has been hindered by the dependence on billions of exploratory interactions via online reinforcement learning, and domain-specific model designs that make...
5.1K
Hiroki Furuta
@frt03_
Feb 5, 2025
拡散モデルのInference-time Alignmentに関するPreprintを公開しました🎉 各denoising stepで、最終的なデータ点の報酬評価に基づく潜在変数のビームサーチを行い、より好ましい動画をtext-to-videoで生成します。計算コストに対するスケーリングが見られます。東大松尾・岩澤研での研究成果です
Hiroki Furuta
@frt03_
Feb 5, 2025
Our new preprint on inference-time alignment for diffusion models is out🎉 To improve text-to-video generation, we propose Diffusion Latent Beam Search with Lookahead Estimator. We should allocate the computation as: Lookahead Steps >> Search Budget >> Denoising Steps 1/8
GIF
8.6K