Latest papers: Difference between revisions

From Robowaifu Institute of Technology
Jump to navigation Jump to search
(→‎Unsorted: /robowaifu/ paper dump)
Line 41: Line 41:
* [https://arxiv.org/abs/2304.14108 DataComp: In search of the next generation of multimodal datasets]
* [https://arxiv.org/abs/2304.14108 DataComp: In search of the next generation of multimodal datasets]
* [https://arxiv.org/abs/2304.14402 LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions]
* [https://arxiv.org/abs/2304.14402 LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions]
* [https://arxiv.org/abs/2304.13653v1 Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement  Learning]
* [https://arxiv.org/abs/2304.13705 Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware]
* [https://arxiv.org/abs/2304.13705 Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware]
* [https://arxiv.org/abs/2304.13013 Stable and low-precision training for large-scale vision-language models]
* [https://arxiv.org/abs/2304.13013 Stable and low-precision training for large-scale vision-language models]
Line 49: Line 50:
* [https://arxiv.org/abs/2304.10970 Can GPT-4 Perform Neural Architecture Search?]
* [https://arxiv.org/abs/2304.10970 Can GPT-4 Perform Neural Architecture Search?]
* [https://arxiv.org/abs/2304.10592 MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models]
* [https://arxiv.org/abs/2304.10592 MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models]
* [https://arxiv.org/abs/2304.08466v1 Synthetic Data from Diffusion Models Improves ImageNet Classification]
* [https://arxiv.org/abs/2304.08460 LongForm: Optimizing Instruction Tuning for Long Text Generation with Corpus Extraction]
* [https://arxiv.org/abs/2304.08460 LongForm: Optimizing Instruction Tuning for Long Text Generation with Corpus Extraction]
* [https://arxiv.org/abs/2304.07327 OpenAssistant Conversations -- Democratizing Large Language Model Alignment]
* [https://arxiv.org/abs/2304.07327 OpenAssistant Conversations -- Democratizing Large Language Model Alignment]
* [https://arxiv.org/abs/2304.06939 Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved With Text]
* [https://arxiv.org/abs/2304.06939 Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved With Text]
* [https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM]
* [https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM]
==== March 2023 ====
* [https://arxiv.org/abs/2303.16199v1 LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention]
* [https://arxiv.org/abs/2303.12712v5 Sparks of Artificial General Intelligence: Early experiments with GPT-4]


==== Februrary 2023 ====
==== Februrary 2023 ====
Line 60: Line 67:
* [https://arxiv.org/abs/2302.13971 LLaMA: Open and Efficient Foundation Language Models]
* [https://arxiv.org/abs/2302.13971 LLaMA: Open and Efficient Foundation Language Models]
* [https://arxiv.org/abs/2302.13939 SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks]
* [https://arxiv.org/abs/2302.13939 SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks]
* [https://arxiv.org/abs/2302.12353 Autonomous Restructuring of Asteroids into Rotating Space Stations]
* [https://arxiv.org/abs/2302.06868 SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains]
* [https://arxiv.org/abs/2302.06868 SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains]
 
* [https://arxiv.org/abs/2302.06675v3 Symbolic Discovery of Optimization Algorithms]
* [https://arxiv.org/abs/2302.12353 Autonomous Restructuring of Asteroids into Rotating Space Stations]
* [https://arxiv.org/abs/2302.01925v1 Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers]
* [https://arxiv.org/abs/2302.00923v4 Multimodal Chain-of-Thought Reasoning in Language Models]


==== January 2023 ====
==== January 2023 ====
Line 70: Line 79:
* [https://arxiv.org/abs/2301.04589 Memory Augmented Large Language Models are Computationally Universal]
* [https://arxiv.org/abs/2301.04589 Memory Augmented Large Language Models are Computationally Universal]
* [https://arxiv.org/abs/2301.04246 Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations]
* [https://arxiv.org/abs/2301.04246 Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations]
* [https://arxiv.org/abs/2301.02111v1 Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers]
* [https://arxiv.org/abs/2301.00774v3 SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot]
* [https://arxiv.org/abs/2301.00303v1 Rethinking with Retrieval: Faithful Large Language Model Inference]


==== December 2022 ====
==== December 2022 ====


* [https://arxiv.org/abs/2212.10560v1 Self-Instruct: Aligning Language Model with Self Generated Instructions]
* [https://arxiv.org/abs/2212.09689 Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor]
* [https://arxiv.org/abs/2212.08751v1 Point-E: A System for Generating 3D Point Clouds from Complex Prompts]
* [https://arxiv.org/abs/2212.08073 Constitutional AI: Harmlessness from AI Feedback]
* [https://arxiv.org/abs/2212.08073 Constitutional AI: Harmlessness from AI Feedback]
* [https://arxiv.org/abs/2212.09689 Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor]
* [https://arxiv.org/abs/2212.08051v1 Objaverse: A Universe of Annotated 3D Objects]
* [https://arxiv.org/abs/2212.03848v2 NeRFEditor: Differentiable Style Decomposition for Full 3D Scene Editing]


==== November 2022 ====
==== November 2022 ====


* [https://arxiv.org/abs/2211.11602 Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback]
* [https://arxiv.org/abs/2211.11602 Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback]
* [https://arxiv.org/abs/2211.10438 SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models]
* [https://arxiv.org/abs/2211.09119v2 Token Turing Machines]
* [https://arxiv.org/abs/2211.06127v1 English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddings]


==== October 2022 ====
==== October 2022 ====
Line 84: Line 103:
* [https://arxiv.org/abs/2210.12217 Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning]
* [https://arxiv.org/abs/2210.12217 Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning]
* [https://arxiv.org/abs/2210.11416 Scaling Instruction-Finetuned Language Models]
* [https://arxiv.org/abs/2210.11416 Scaling Instruction-Finetuned Language Models]
* [https://arxiv.org/abs/2210.11948v2 lo-fi: distributed fine-tuning without communication]
* [https://arxiv.org/abs/2210.11610v2 Large Language Models Can Self-Improve]
* [https://arxiv.org/abs/2210.06407v1 Interactive Language: Talking to Robots in Real Time]
* [https://arxiv.org/abs/2210.06407v1 Interactive Language: Talking to Robots in Real Time]
* [https://arxiv.org/abs/2210.05836v1 CLIP also Understands Text: Prompting CLIP for Phrase Understanding]


==== September 2022 ====
==== September 2022 ====
Line 89: Line 113:
* [https://arxiv.org/abs/2209.15189 Learning by Distilling Context]
* [https://arxiv.org/abs/2209.15189 Learning by Distilling Context]
* [https://arxiv.org/abs/2209.07662 Dynamic Generation of Interpretable Inference Rules in a Neuro-Symbolic Expert System]
* [https://arxiv.org/abs/2209.07662 Dynamic Generation of Interpretable Inference Rules in a Neuro-Symbolic Expert System]
* [https://arxiv.org/abs/2209.10655v3 Mega: Moving Average Equipped Gated Attention]
* [https://arxiv.org/abs/2209.00626v4 The alignment problem from a deep learning perspective]


==== August 2022 ====
==== August 2022 ====


* [https://arxiv.org/abs/2208.07339 LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale]
* [https://arxiv.org/abs/2208.07339 LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale]
* [https://arxiv.org/abs/2208.01448v2 AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model]
* [https://arxiv.org/abs/2208.00748v3 Efficient Long-Text Understanding with Short-Text Models]
==== July 2022 ====
* [https://arxiv.org/abs/2207.07061v2 Confident Adaptive Language Modeling]
* [https://arxiv.org/abs/2207.05608v1 Inner Monologue: Embodied Reasoning through Planning with Language Models]
==== June 2022 ====
* [https://arxiv.org/abs/2206.15472v3 On-Device Training Under 256KB Memory]
* [https://arxiv.org/abs/2206.11795v1 Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos]
* [https://arxiv.org/abs/2206.07568v2 Contrastive Learning as Goal-Conditioned Reinforcement Learning]
* [https://arxiv.org/abs/2206.04615v2 Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models]
* [https://arxiv.org/abs/2206.02690v3 A Survey on Sentence Embedding Models Performance for Patent Analysis]
* [https://arxiv.org/abs/2206.00843v2 DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks]


==== May 2022 ====
==== May 2022 ====


* [https://arxiv.org/abs/2205.14135v2 FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness]
* [https://arxiv.org/abs/2205.14100v5 GIT: A Generative Image-to-text Transformer for Vision and Language]
* [https://arxiv.org/abs/2205.12910 NaturalProver: Grounded Mathematical Proof Generation with Language Models]
* [https://arxiv.org/abs/2205.12910 NaturalProver: Grounded Mathematical Proof Generation with Language Models]
* [https://arxiv.org/abs/2205.11916v4 Large Language Models are Zero-Shot Reasoners]
* [https://arxiv.org/abs/2205.06175 A Generalist Agent]
* [https://arxiv.org/abs/2205.06175 A Generalist Agent]
* [https://arxiv.org/abs/2205.05270v1 Relational Triple Extraction: One Step is Enough]
* [https://arxiv.org/abs/2205.05131v3 UL2: Unifying Language Learning Paradigms]
* [https://arxiv.org/abs/2205.05131v3 UL2: Unifying Language Learning Paradigms]
* [https://arxiv.org/abs/2205.05055v6 Data Distributional Properties Drive Emergent In-Context Learning in Transformers]
* [https://arxiv.org/abs/2205.01917v2 CoCa: Contrastive Captioners are Image-Text Foundation Models]s]
* [https://arxiv.org/abs/2205.01917 CoCa: Contrastive Captioners are Image-Text Foundation Models]


==== March 2022 ====
==== March 2022 ====


* [https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html In-context Learning and Induction Heads]
* [https://arxiv.org/abs/2203.13474 CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis]
* [https://arxiv.org/abs/2203.13474 CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis]
* [https://arxiv.org/abs/2203.02155 Training language models to follow instructions with human feedback]
* [https://arxiv.org/abs/2203.02155 Training language models to follow instructions with human feedback]
* [https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html In-context Learning and Induction Heads]
* [https://arxiv.org/abs/2203.16329v2 Parameter-efficient Model Adaptation for Vision Transformers]
* [https://arxiv.org/abs/2203.15556v1 Training Compute-Optimal Large Language Models]
* [https://arxiv.org/abs/2203.07852v3 Block-Recurrent Transformers]
* [https://arxiv.org/abs/2203.06904v2 Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models]
* [https://arxiv.org/abs/2203.03466v2 Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer]
* [https://arxiv.org/abs/2203.02155v1 Training language models to follow instructions with human feedback]
 
==== February 2022 ====
 
* [https://arxiv.org/abs/2202.12205v2 Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review]
* [https://arxiv.org/abs/2202.10447v2 Transformer Quality in Linear Time]
* [https://arxiv.org/abs/2202.06991v3 Transformer Memory as a Differentiable Search Index]
 
==== January 2022 ====
 
* [https://arxiv.org/abs/2201.12122v3 Can Wikipedia Help Offline Reinforcement Learning?]
* [https://arxiv.org/abs/2201.11903v6 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models]
* [https://arxiv.org/abs/2201.06910v2 ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization]


==== Februrary 2022 ====
==== Februrary 2022 ====
Line 114: Line 182:
* [https://transformer-circuits.pub/2021/framework/index.html A Mathematical Framework for Transformer Circuits]
* [https://transformer-circuits.pub/2021/framework/index.html A Mathematical Framework for Transformer Circuits]
* [https://arxiv.org/abs/2112.05682 Self-attention Does Not Need <math>O(n^2)</math> Memory]
* [https://arxiv.org/abs/2112.05682 Self-attention Does Not Need <math>O(n^2)</math> Memory]
* [https://arxiv.org/abs/2112.05253v2 MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuning]
* [https://arxiv.org/abs/2112.04426v3 Improving language models by retrieving from trillions of tokens]
* [https://arxiv.org/abs/2112.03254v3 Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention]
==== November 2021 ====
* [https://arxiv.org/abs/2111.05204v1 Reason first, then respond: Modular Generation for Knowledge-infused Dialogue]
* [https://arxiv.org/abs/2111.02114v1 LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs]


==== October 2021 ====
==== October 2021 ====


* [https://arxiv.org/abs/2110.08207v3 Multitask Prompted Training Enables Zero-Shot Task Generalization]
* [https://arxiv.org/abs/2110.07732 The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization]
* [https://arxiv.org/abs/2110.07732 The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization]
* [https://arxiv.org/abs/2110.05651v2 Learning with Algorithmic Supervision via Continuous Relaxations]
* [https://arxiv.org/abs/2110.00296 Powerpropagation: A sparsity inducing weight reparameterisation]
* [https://arxiv.org/abs/2110.00296 Powerpropagation: A sparsity inducing weight reparameterisation]
* [https://arxiv.org/abs/2111.00210 Mastering Atari Games with Limited Data]


==== September 2021 ====
==== September 2021 ====


* [https://arxiv.org/abs/2109.14076v3 RAFT: A Real-World Few-Shot Text Classification Benchmark]
* [https://arxiv.org/abs/2109.10686v2 Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers]
* [https://arxiv.org/abs/2109.09193v1 Towards Zero-Label Language Learning]
* [https://arxiv.org/abs/2109.08603 Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration]
* [https://arxiv.org/abs/2109.08603 Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration]
* [https://arxiv.org/abs/2109.06668v6 Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain]
* [https://arxiv.org/abs/2109.01652v5 Finetuned Language Models Are Zero-Shot Learners]
* [https://arxiv.org/abs/2109.00157v2 A Survey of Exploration Methods in Reinforcement Learning]
==== August 2021 ====
* [https://arxiv.org/abs/2108.01077v3 Generating Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution]
==== June 2021 ====
* [https://arxiv.org/abs/2106.15515v4 What Is Consciousness? Artificial Intelligence, Real Intelligence, Quantum Mind, And Qualia]
* [https://arxiv.org/abs/2106.13884v2 Multimodal Few-Shot Learning with Frozen Language Models]
* [https://arxiv.org/abs/2106.09685v2 LoRA: Low-Rank Adaptation of Large Language Models]
* [https://arxiv.org/abs/2106.06295v2 Going Beyond Linear Transformers with Recurrent Fast Weight Programmers]
* [https://arxiv.org/abs/2106.04647v2 Compacter: Efficient Low-Rank Hypercomplex Adapter Layers]
* [https://arxiv.org/abs/2106.04489v1 Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks]
* [https://arxiv.org/abs/2106.00882v1 Efficient Passage Retrieval with Hashing for Open-domain Question Answering]
* [https://arxiv.org/abs/2106.00874v2 Conversational Question Answering: A Survey]
==== May 2021 ====
* [https://arxiv.org/abs/2105.13290v3 CogView: Mastering Text-to-Image Generation via Transformers]
* [https://arxiv.org/abs/2105.13290v3 CogView: Mastering Text-to-Image Generation via Transformers]
* [https://arxiv.org/abs/2105.10311v2 Pretrained Language Models for Text Generation: A Survey]
* [https://arxiv.org/abs/2105.01883v3 RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition]
==== April 2021 ====
* [https://arxiv.org/abs/2104.14294v2 Emerging Properties in Self-Supervised Vision Transformers]
* [https://arxiv.org/abs/2104.12763v2 MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding]
* [https://arxiv.org/abs/2104.06303v1 Learning and Planning in Complex Action Spaces]
* [https://arxiv.org/abs/2104.06069v2 1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed]
* [https://arxiv.org/abs/2104.05565v3 Survey on reinforcement learning for language processing]
* [https://arxiv.org/abs/2104.04644v3 Fast and Efficient Locomotion via Learned Gait Transitions]
* [https://arxiv.org/abs/2104.00298v3 EfficientNetV2: Smaller Models and Faster Training]
==== March 2021 ====
* [https://arxiv.org/abs/2103.12656v2 Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classification]
* [https://arxiv.org/abs/2103.12407v4 Detecting Hate Speech with GPT-3]
* [https://arxiv.org/abs/2103.11955v3 Improving and Simplifying Pattern Exploiting Training]
* [https://arxiv.org/abs/2103.08541v1 Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence]
* [https://arxiv.org/abs/2103.00020v1 Learning Transferable Visual Models From Natural Language Supervision]
==== February 2021 ====
* [https://arxiv.org/abs/2102.11174v3 Linear Transformers Are Secretly Fast Weight Programmers]
==== January 2021 ====
* [https://arxiv.org/abs/2101.03961v3 Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity]


==== December 2020 ====
==== December 2020 ====


* [https://arxiv.org/abs/2012.08508v3 Attention over learned object embeddings enables complex visual reasoning]
* [https://arxiv.org/abs/2012.06884 AIR-FI: Generating Covert Wi-Fi Signals from Air-Gapped Computers]
* [https://arxiv.org/abs/2012.06884 AIR-FI: Generating Covert Wi-Fi Signals from Air-Gapped Computers]
* [https://arxiv.org/abs/2012.05876v2 Neurosymbolic AI: The 3rd Wave]
==== November 2020 ====
* [https://arxiv.org/abs/2011.09284v2 3D imaging from multipath temporal echoes]
* [https://arxiv.org/abs/2011.06392v2 Using IPA-Based Tacotron for Data Efficient Cross-Lingual Speaker Adaptation and Pronunciation Enhancement]
* [https://arxiv.org/abs/2011.05315v2 Is Private Learning Possible with Instance Encoding?]
==== October 2020 ====
* [https://arxiv.org/abs/2010.11967v1 Language Models are Open Knowledge Graphs]
* [https://arxiv.org/abs/2010.06775v1 Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision]
==== September 2020 ====
* [https://arxiv.org/abs/2009.07118v2 It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners]
* [https://arxiv.org/abs/2009.06489v2 The Hardware Lottery]
==== August 2020 ====
* [https://arxiv.org/abs/2008.02268v3 NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections]
==== July 2020 ====
* [https://arxiv.org/abs/2007.08794v3 Discovering Reinforcement Learning Algorithms]


==== September 2020 ====
==== September 2020 ====
Line 135: Line 294:


* [https://arxiv.org/abs/2006.15191 Is SGD a Bayesian sampler? Well, almost]
* [https://arxiv.org/abs/2006.15191 Is SGD a Bayesian sampler? Well, almost]
* [https://arxiv.org/abs/2006.08517v1 The Limit of the Batch Size]
* [https://arxiv.org/abs/2006.07549v2 Hindsight Expectation Maximization for Goal-conditioned Reinforcement Learning]
* [https://arxiv.org/abs/2006.04768 Linformer: Self-Attention with Linear Complexity]
* [https://arxiv.org/abs/2006.04182v5 Predictive Coding Approximates Backprop along Arbitrary Computation Graphs]
==== May 2020 ====
* [https://arxiv.org/abs/2005.14165v4 Language Models are Few-Shot Learners]
* [https://arxiv.org/abs/2005.13213v1 Give Me Convenience and Give Her Death: Who Should Decide What Uses of NLP are Appropriate, and on What Basis?]
* [https://arxiv.org/abs/2005.07327v2 ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language]
* [https://arxiv.org/abs/2005.05957v3 Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis]
* [https://arxiv.org/abs/2005.05106v2 Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech]
==== April 2020 ====
* [https://arxiv.org/abs/2004.15011v3 TLDR: Extreme Summarization of Scientific Documents]
* [https://arxiv.org/abs/2004.13637v2 Recipes for building an open-domain chatbot]
==== March 2020 ====
* [https://arxiv.org/abs/2003.06965v1 OmniTact: A Multi-Directional High Resolution Touch Sensor]
* [https://arxiv.org/abs/2003.04887v2 ReZero is All You Need: Fast Convergence at Large Depth]
==== February 2020 ====
* [https://arxiv.org/abs/2002.11794v2 Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers]
* [https://arxiv.org/abs/2002.10585v1 Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity]
* [https://arxiv.org/abs/2002.09571v2 Learning to Continually Learn]
* [https://arxiv.org/abs/2002.00388v4 A Survey on Knowledge Graphs: Representation, Acquisition and Applications]


==== January 2020 ====
==== January 2020 ====


* [https://arxiv.org/abs/2001.04063 ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training]
* [https://arxiv.org/abs/2001.04063 ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training]
* [https://arxiv.org/abs/2001.09977v3 Towards a Human-like Open-Domain Chatbot]
* [https://arxiv.org/abs/2001.07676v3 Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference]
* [https://arxiv.org/abs/2001.06782v4 Gradient Surgery for Multi-Task Learning]
* [https://arxiv.org/abs/2001.04463v3 Unsupervised Audiovisual Synthesis via Exemplar Autoencoders]
==== December 2019 ====
* [https://arxiv.org/abs/1912.07768v1 Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data]
* [https://arxiv.org/abs/1912.02315v2 12-in-1: Multi-Task Vision and Language Representation Learning]
* [https://arxiv.org/abs/1912.01412v1 Deep Learning for Symbolic Mathematics]
* [https://arxiv.org/abs/1912.01320v1 ATIS + SpiNNaker: a Fully Event-based Visual Tracking Demonstration]
==== November 2019 ====
* [https://arxiv.org/abs/1911.08265v2 Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model]
==== October 2019 ====
* [https://arxiv.org/abs/1910.14238v1 Learning Disentangled Representations for Recommendation]
* [https://arxiv.org/abs/1910.06764v1 Stabilizing Transformers for Reinforcement Learning]
* [https://arxiv.org/abs/1910.06711v3 MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis]
* [https://arxiv.org/abs/1910.03175v5 MIM: Mutual Information Machine]
==== September 2019 ====
* [https://arxiv.org/abs/1909.08593v2 Fine-Tuning Language Models from Human Preferences]
* [https://arxiv.org/abs/1909.03341v2 Neural Machine Translation with Byte-Level Subwords]
==== July 2019 ====
* [https://arxiv.org/abs/1907.05242v2 Large Memory Layers with Product Keys]
* [https://arxiv.org/abs/1907.03976v3 Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations]
==== June 2019 ====
* [https://arxiv.org/abs/1906.05833v2 There is no Artificial General Intelligence]
==== May 2019 ====
* [https://arxiv.org/abs/1905.11946v5 EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks]
* [https://arxiv.org/abs/1905.07579v1 Combining Experience Replay with Exploration by Random Network Distillation]
==== April 2019 ====
* [https://arxiv.org/abs/1904.10278v2 Improving Differentiable Neural Computers Through Memory Masking, De-allocation, and Link Distribution Sharpness Control]
* [https://arxiv.org/abs/1904.00962v5 Large Batch Optimization for Deep Learning: Training BERT in 76 minutes]
==== January 2019 ====
* [https://arxiv.org/abs/1901.10995v4 Go-Explore: a New Approach for Hard-Exploration Problems]
* [https://arxiv.org/abs/1901.01753v3 Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions]
==== November 2018 ====
* [https://arxiv.org/abs/1811.12927v2 Hierarchical Policy Design for Sample-Efficient Learning of Robot Table Tennis Through Self-Play]
* [https://arxiv.org/abs/1811.11742v2 3D human pose estimation in video with temporal convolutions and semi-supervised training]
* [https://arxiv.org/abs/1811.00945v2 Image Chat: Engaging Grounded Conversations]
* [https://arxiv.org/abs/1811.00207v5 Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset]
==== October 2018 ====
* [https://arxiv.org/abs/1810.12894v1 Exploration by Random Network Distillation]
* [https://arxiv.org/abs/1810.12217v1 Dreaming neural networks: forgetting spurious memories and reinforcing pure ones]
* [https://arxiv.org/abs/1810.04805v2 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding]
==== September 2018 ====
* [https://arxiv.org/abs/1809.09600v1 HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering]
* [https://arxiv.org/abs/1809.08267v3 Neural Approaches to Conversational AI]
==== July 2018 ====
* [https://arxiv.org/abs/1807.08107v1 Person Search via A Mask-Guided Two-Stream CNN Model]
==== June 2018 ====
* [https://arxiv.org/abs/1806.03822v1 Know What You Don't Know: Unanswerable Questions for SQuAD]
==== April 2018 ====
* [https://arxiv.org/abs/1804.02464v3 Differentiable plasticity: training plastic neural networks with backpropagation]
* [https://arxiv.org/abs/1804.01756v3 The Kanerva Machine: A Generative Distributed Memory]
==== March 2018 ====
* [https://arxiv.org/abs/1904.06736v1 A Short Survey On Memory Based Reinforcement Learning]
* [https://arxiv.org/abs/1803.11175v2 Universal Sentence Encoder]
* [https://arxiv.org/abs/1803.10760 Unsupervised Predictive Memory in a Goal-Directed Agent]
* [https://arxiv.org/abs/1803.10122v4 World Models]
==== February 2018 ====
* [https://arxiv.org/abs/1802.01561v3 IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures]
==== January 2018 ====
* [https://arxiv.org/abs/1810.10222v1 Universal Language Model Fine-Tuning with Subword Tokenization for Polish]
* [https://arxiv.org/abs/1801.07243v5 Personalizing Dialogue Agents: I have a dog, do you have pets too?]
* [https://arxiv.org/abs/1801.05667v1 Innateness, AlphaZero, and Artificial Intelligence]
==== December 2017 ====
* [https://arxiv.org/abs/1712.05884v2 Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions]
* [https://arxiv.org/abs/1712.02950v2 CycleGAN, a Master of Steganography]
* [https://arxiv.org/abs/1712.01815v1 Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm]
==== November 2017 ====
* [https://arxiv.org/abs/1711.00489v2 Don't Decay the Learning Rate, Increase the Batch Size]
==== October 2017 ====
* [https://arxiv.org/abs/1710.05941v2 Searching for Activation Functions]
* [https://arxiv.org/abs/1710.03957v1 DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset]
==== September 2017 ====
* [https://arxiv.org/abs/1709.07871v2 FiLM: Visual Reasoning with a General Conditioning Layer]
==== August 2017 ====
* [https://arxiv.org/abs/1808.04293v1 Fast, Better Training Trick -- Random Gradient]
* [https://arxiv.org/abs/1708.02190v3 Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning]
==== July 2017 ====
* [https://arxiv.org/abs/1707.01495v3 Hindsight Experience Replay]
==== June 2017 ====
* [https://arxiv.org/abs/2002.05202v1 GLU Variants Improve Transformer]
==== May 2017 ====
* [https://arxiv.org/abs/1705.03551v2 TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension]
==== March 2017 ====
* [https://arxiv.org/abs/1703.10135v2 Tacotron: Towards End-to-End Speech Synthesis]
==== November 2016 ====
* [https://arxiv.org/abs/1611.08669v5 Visual Dialog]
* [https://arxiv.org/abs/1611.04558v2 Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation]
==== October 2016 ====
* [https://arxiv.org/abs/1610.00956v1 Embracing data abundance: BookTest Dataset for Reading Comprehension]
==== September 2016 ====
* [https://arxiv.org/abs/1609.04836v2 On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima]
* [https://arxiv.org/abs/1609.02228v2 Learning to learn with backpropagation of Hebbian plasticity]
==== July 2016 ====
* [https://arxiv.org/abs/1607.06450v1 Layer Normalization]
==== June 2016 ====
* [https://arxiv.org/abs/1606.08415v4 Gaussian Error Linear Units (GELUs)]
* [https://arxiv.org/abs/1606.06565v2 Concrete Problems in AI Safety]
==== May 2016 ====
* [https://arxiv.org/abs/1605.07683v4 Learning End-to-End Goal-Oriented Dialog]
==== November 2015 ====
* [https://arxiv.org/abs/1511.02301v4 The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations]
==== June 2015 ====
* [https://arxiv.org/abs/1506.08909v3 The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems]
* [https://arxiv.org/abs/1506.03340v3 Teaching Machines to Read and Comprehend]
==== May 2015 ====
* [https://arxiv.org/abs/1505.06366v2 Open Ended Intelligence: The individuation of Intelligent Agents]
==== June 2015 ====
* [https://arxiv.org/abs/1506.08909v3 The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems]
* [https://arxiv.org/abs/1506.03340v3 Teaching Machines to Read and Comprehend]
==== July 2014 ====
* [https://arxiv.org/abs/1407.3501v4 Robots that can adapt like animals]


==== December 2012 ====
==== December 2012 ====

Revision as of 00:58, 5 May 2023

This page requires tidying up!
This page needs to be completely reformatted. Will be changing the tl;drs to be title text so you can hover over links to get the gist of them without clicking.
This page requires expansion!
This page needs papers! Probably should set up an automated system so I can just drop Twitter and Arxiv links.

This page serves to collect notable research papers related to robotics and artificial intelligence, particularly ones that can be used by hobbyists with minimal resources towards creating robowaifus. Feel free to add new papers to the list and discuss any papers on the talk page. Papers posted on /robowaifu/ will also eventually appear here.

Search sites

Social media sources

2023

Unsorted

Need to summarize these papers into tl;dr. An automated system for this would be great.

May 2023

April 2023

March 2023

Februrary 2023

January 2023

December 2022

November 2022

October 2022

September 2022

August 2022

July 2022

June 2022

May 2022


March 2022

February 2022

January 2022

Februrary 2022

December 2021

November 2021

October 2021

September 2021

August 2021

June 2021

May 2021

April 2021

March 2021

February 2021

January 2021

December 2020

November 2020

October 2020

September 2020

August 2020

July 2020

September 2020

June 2020

May 2020

April 2020

March 2020

February 2020

January 2020

December 2019

November 2019

October 2019

September 2019

July 2019

June 2019

May 2019

April 2019

January 2019

November 2018

October 2018

September 2018

July 2018

June 2018

April 2018

March 2018

February 2018

January 2018

December 2017

November 2017

October 2017

September 2017

August 2017

July 2017

June 2017

May 2017

March 2017

November 2016

October 2016

September 2016

July 2016

June 2016

May 2016

November 2015

June 2015

May 2015

June 2015

July 2014

December 2012

September 2003

Instruction tuning

Evol-Instruct: Mass-Producing Open-Domain Instruction Data with Varying Levels of Complexity using Large Language Models (arXiv:2304.12244)

tl;dr The paper proposes a method called Evol-Instruct for creating large amounts of instruction data with different levels of complexity using a large language model (LLM) instead of humans. The generated data is used to fine-tune another LLM called WizardLM. Human evaluations show that Evol-Instruct instructions are better than human-created ones, and WizardLM is preferred over OpenAI ChatGPT for complex tasks. The study suggests that fine-tuning LLMs with AI-evolved instructions is a promising approach for improving their performance.[1]


2022

November 2022

Large Language Models Are Human-Level Prompt Engineers (arXiv)

tl;dr OpenReview version. Automatic Prompt Engineer (APE) is a method that can generate instructions automatically. It uses a pool of generated instruction candidates and evaluates the quality of them by the zero-shot performance of another LLM following a selected instruction.[2]


2021

PROTIP: You can use sshleifer/distilbart-cnn-12-6 to help with summarizing papers. Check the paper template for usage instructions.
2023 update: Leaving this note here as a relic of how much things have progressed. PROTIP: Use GPT-4.

August 2021

Computer vision

NeuralMVS: Bridging Multi-View Stereo and Novel View Synthesis (arXiv:2108.03880)

tl;dr Multi-view stereo is a core task in 3D computer vision. NeRF methods do not generalize to novel scenes and are slow to train and test. We propose to bridge the gap between these two methodologies with a novel network that can recover 3D scene geometry as a distance function.[3]


Simulation

iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks (arXiv:2108.03272)

tl;dr iGibson 2.0 is a novel simulation environment using Bullet that supports the simulation of a more diverse set of household tasks through three key innovations. Firstly, it supports object states, including temperature, wetness level, cleanliness level, and toggled and sliced states, necessary to cover a wider range of tasks. Second, it implements a set of predicate logic functions that map the simulator states to logic states like Cooked or Soaked. Third, the simulator can sample valid physical states that satisfy a logic state. This functionality can generate potentially infinite instances of tasks with minimal effort from the users.[4]


July 2021

Audio processing

SoundStream: An End-to-End Neural Audio Codec (arXiv:2107.03312)

tl;dr A novel neural audio codec that can efficiently compress speech, music and general audio at bitrates normally targeted by speech-tailored codecs. SoundStream at 3kbps outperforms Opus at 12kbps and approaches EVS at 9.6kbps.[5]


June 2021

Multimodal learning

Multimodal Few-Shot Learning with Frozen Language Models (arXiv:2106.13884)

tl;dr When trained at sufficient scale, auto-regressive language models exhibit the notable ability to learn a new language task after being prompted with just a few examples. Here, the authors present a simple, yet effective, approach for transferring this few-shot learning ability to a multimodal setting (vision and language).[6]


Optimizers

A Generalizable Approach to Learning Optimizers (arXiv:2106.00958)

tl;dr Learning to update optimizer hyperparameters instead of model parameters directly using novel features, actions, and a reward function.[7]


May 2021

Memory

Not All Memories are Created Equal: Learning to Forget by Expiring (arXiv:2105.06548)

tl;dr The authors propose Expire-Span, a method that learns to retain the most important information and expire the irrelevant information, which enables Transformers to scale to attend over tens of thousands of previous timesteps efficiently.[8]


April 2021

Fine-tuning

The Power of Scale for Parameter-Efficient Prompt Tuning (arXiv:2104.08691)

tl;dr In this work, the author's explore "prompt tuning" a simple but effective mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks.[9]


March 2021

Computer vision

NeX: Real-time View Synthesis with Neural Basis Expansion (arXiv:2103.05606)

tl;dr The authors present NeX, a new approach to novel view synthesis based on enhancements of multiplane image (MPI) that can reproduce next-level view-dependent effects -- in real time. The method achieves the best overall scores across all major metrics on these datasets with more than 1000× faster rendering time than the state of the art.[10]


October 2020

Computer vision

GRF: Learning a General Radiance Field for 3D Scene Representation and Rendering (arXiv:2010.04595)

tl;dr General Radiance Fields construct an internal representation for each 3D point of a scene from 2D inputs and renders the corresponding appearance and geometry of any 3D scene viewing from an arbitrary angle.[11]


September 2020

Summarization

Learning to Summarize with Human Feedback (arXiv:2009.01325)

tl;dr Human feedback models outperform much larger supervised models and reference summaries on TL;DR.[12]


December 2019

Meta-learning

Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data (arXiv:1912.07768)

tl;dr This paper investigates the intriguing question of whether learning algorithms can automatically generate training data, learning environments, and curricula in order to help AI agents rapidly learn. GTNs are deep neural networks that generate data and/or training environments that a learner trains on for a few SGD steps before being tested on a target task. It then differentiates through the entire learning process via meta-gradients to update the GTN parameters to improve performance on the target task.[13]


Older papers

See also

References

  1. ↑ Xu et al. Evol-Instruct: Mass-Producing Open-Domain Instruction Data with Varying Levels of Complexity using Large Language Models. arXiv:2304.12244, 2023.
  2. ↑ Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, Jimmy Ba. Large Language Models Are Human-Level Prompt Engineers. arXiv, 2022.
  3. ↑ Radu Alexandru Rosu, Sven Behnke. NeuralMVS: Bridging Multi-View Stereo and Novel View Synthesis. arXiv:2108.03880, 2021.
  4. ↑ Chengshu Li, Fei Xia, Roberto Martín-Martín, Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, Andrey Kurenkov, Karen Liu, Hyowon Gweon, Jiajun Wu, Li Fei-Fei, Silvio Savarese. iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks. arXiv:2108.03272, 2021.
  5. ↑ Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, Marco Tagliasacchi. SoundStream: An End-to-End Neural Audio Codec. arXiv:2107.03312, 2021.
  6. ↑ Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, S. M. Ali Eslami, Oriol Vinyals, Felix Hill. Multimodal Few-Shot Learning with Frozen Language Models. arXiv:2106.13884, 2021.
  7. ↑ Diogo Almeida, Clemens Winter, Jie Tang, Wojciech Zaremba. A Generalizable Approach to Learning Optimizers. arXiv:2106.00958, 2021.
  8. ↑ Sainbayar Sukhbaatar, Da Ju, Spencer Poff, Stephen Roller, Arthur Szlam, Jason Weston, Angela Fan. Not All Memories are Created Equal: Learning to Forget by Expiring. arXiv:2105.06548, 2021.
  9. ↑ Brian Lester, Rami Al-Rfou, Noah Constant. The Power of Scale for Parameter-Efficient Prompt Tuning. arXiv:2104.08691, 2021.
  10. ↑ Suttisak Wizadwongsa, Pakkapon Phongthawee, Jiraphon Yenphraphai, Supasorn Suwajanakorn. NeX: Real-time View Synthesis with Neural Basis Expansion. arXiv:2103.05606, 2021.
  11. ↑ Alex Trevithick, Bo Yang. GRF: Learning a General Radiance Field for 3D Scene Representation and Rendering. arXiv:2010.04595, 2020.
  12. ↑ Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, Paul Christiano. Learning to Summarize with Human Feedback. arXiv:2009.01325, 2020.
  13. ↑ Felipe Petroski Such, Aditya Rawal, Joel Lehman, Kenneth O. Stanley, Jeff Clune. Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data. arXiv:1912.07768, 2019.
Retrieved from "https://robowaifu.tech/w/index.php?title=Latest_papers&oldid=316"