Self-supervised and unsupervised learning for video
目录
ActBERT: Learning Global-Local Video-Text Representations（cvpr2020）
Temporal Cycle-Consistency Learning（Google Brain）
Unsupervised leaning of visual representation using videos
Learning Video Object Segmentation from Unlabeled Videos
Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
ActBERT: Learning Global-Local Video-Text Representations（cvpr2020）
背景：
- 在教做菜的视频中，视频创作者会描述，“开始切胡萝卜”，往往人物也是在正进行“切胡萝卜”的动作。这种天然的视觉对应关系，是进行视频文字自监督学习的重要要素。其中文字描述可以通过自动语音识别技术（ASR）从视频中或从创作者上传的字幕中提取文字。这样成对的视频文字数据就产生了。
- 为了解决视觉特征无法做分类预测的问题，VideoBERT 使用了 hierachical k-means 的方法将视觉特征进行离散化，这样每个数据特征都对应一个离散的视觉中心。
idea：在输入层同时加入全局动作特征与局部区域特征。
论文链接：https://openaccess.thecvf.com/content_CVPR_2020/papers/Zhu_ActBERT_Learning_Global-Local_Video-Text_Representations_CVPR_2020_paper.pdf
参考：https://www.jiqizhixin.com/articles/2020-07-20-8
Self-supervised learning using consistency regularization of spatio-temporal data augmentation for action recognition（STCR）
分为两个Branch, 一路Clean表示普通的Video,另一路表示引入Noise之后的, 2路不同的输入经过3D Backbone之后，我们希望feature 在 temporal-level和feature-level保持consistency。
Temporal Cycle-Consistency Learning（Google Brain）
TCC的关键在于通过循环一致性的原则，从多个视频中寻找出对应的相同动作。
研究人员首先将两个待配准的视频传入编码器中获取对应的嵌入信息，而后选择两个视频来进行TCC的训练，其中video1作为参考视频，从中取出一帧并利用最邻近方法在嵌入空间中找到video2中最为近似的一帧；而后以video2中找到的这一帧作为输入，从新到嵌入空间中寻找video中对应的帧(circle过程)。如果学习出的嵌入空间具有循环一致性的话，这一帧和先前输入video1的参考帧应该是同一帧。
参考：https://zhuanlan.zhihu.com/p/77771815
Unsupervised leaning of visual representation using videos
idea:用一个图片（patch）三元组（a, b, c）来训练网络，其中a和b是同一个视频中内容相近的patch， c是从其他视频中任意采样得到的patch，让网络能学习到a, b, c的特征并满足a,b之间的距离尽量小，a,c之间的距离尽量大。
参考：blog
Learning Video Object Segmentation from Unlabeled Videos(cvpr2020)
Author: Xiankai Lu...Steven C. H. Hoi
Key: 提出了一种无监督方法，从四个方面进行特征学习。类似于几个proxy task的组合。包括：frame, short-term, long-term and whole video granularities
Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
论文：https://arxiv.org/pdf/1811.09795.pdf
Other methods
- Video Generation with GAN
与图像类生成方法类似。这种方法通常由两部分组成：generator（生成器：生成视频）+ discriminator（鉴别器：区分生成的与真实的视频）。Discriminator可以被迁移物其他的下游任务。
- Video Colorization
给一个灰色的图像，一些参考的彩色图像，让模型学习上色。
-
Video Future Prediction
给定一些序列，让模型学习预测后续的序列。如：Self-Supervised Video Representation Learning With Odd-One-Out Networks
参考：https://bbs.huaweicloud.com/blogs/181241
相关信息
- 《 Self-supervised Sample Mining》笔记
- What is the difference between supervised learning and unsupervised learning?
- Unsupervised Learning: Neighbor Embedding
- 论文解读（S^3-CL）《Structural and Semantic Contrastive Learning for Self-supervised Node Representation Learning》
- Stepwise Metric Promotion for Unsupervised Video Person Re-identification
- Coursera机器学习课程笔记(1) Supervised Learning and Unsupervised Learning
- 【论文读后感】《Self-Attention Enhanced CNNs and Collaborative Curriculum Learning for Distantly Supervised》
- 论文笔记：Evolving Losses for Unsupervised Video Representation Learning
- Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness R
- 论文解读（AutoSSL）《Automated Self-Supervised Learning for Graphs》
- Unsupervised learning无监督学习
- Unsupervised Feature Learning and Deep Learning(UFLDL) Exercise 总结
- CVPR：Weakly-supervised Deep Convolutional Neural Network Learning for Facial Action Intensity Estima
- 论文解读（CSSL）《Contrastive Self-supervised Learning for Graph Classification》
- 监督学习(Supervised learning)
- Boosting self-supervised Learning via Knowledge Transfer
- 论文《Unsupervised Continual Learning And Self-Taught Associative Memory Hierarchies》解读
- 【IROS2018】Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots
- 论文笔记-Self-Supervised Learning With Adaptive Distillation for Hyperspectral Ima
- Self-Supervised Sparse-to-Dense: Self-Supervised Depth Completion from LiDAR and Monocular Camera
- 自监督学习（二）《Scaling and Benchmarking Self-Supervised Visual Representation Learning》
- 论文阅读 《Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis》
- 论文解读（MGAE）《MGAE: Masked Autoencoders for Self-Supervised Learning on Graphs》
- 《Self-Supervised Difference Detection for Weakly-Supervised Semantic Segmentation》笔记
- 每天一篇论文 350/365 MVP: Unified Motion and Visual Self-Supervised Learning for Large-Scale Robotic Navig
- 【论文笔记】CIKM2020 S3-Rec: Self-Supervised Learning for Sequential Recommendation with MIM
- Supervised Learning （监督学习）与 Unsupervised Learning （非监督学习）
- 监督学习（supervised learning）与非监督学习（unsupervised learning）
- free video tutorial of Deep Learning
- Socially-Aware Self-Supervised Tri-Training for Recommendation
- Self-Supervised Learning 的近期发展综述
- Self-Supervised Sparse-to-Dense:Self-Supervised Depth Completion from LiDAR and Monocular Camera
- 论文阅读：Dual Supervised Learning for Natural Language Understanding and Generation
- Unsupervised Learning for Real-World Super-Resolution
- Adversarial Reinforcement Learning for Unsupervised Domain Adaptation
- DetCo: Unsupervised Contrastive Learning for Object Detection
- Momentum Contrast for Unsupervised Visual Representation Learning
- ICRA2019最佳论文 Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representatio
- Dynamic Label Graph Matching for Unsupervised Video Re-Identification
- Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
- 论文阅读：Dual Supervised Learning
- Deeping Learning for self-driving cars
- DEFORM-GAN:AN UNSUPERVISED LEARNING MODEL FOR DEFORMABLE REGISTRATION
- An Unsupervised Autoregressive Model for Speech Representation Learning
- SegFlow_Joint Learning for Video Object Segmentation and Optical Flow 论文翻译
- Unsupervised learning and clustered connectivity enhance reinforcement learning in spiking neural networks
- 人体解析--Look into Person: Self-supervised Structure-sensitive Learning
- python中的self
- Neural Networks and Deep Learning 笔记
- [小样本学习] 论文笔记 Learning to Self-Train for Semi-Supervised Few-Shot Classification
热门资讯
推荐资讯
最新资讯
- 每个程序员必学的10个Git命令
- Android开发之蓝牙详解(一)
- Android开发之蓝牙详解(三)
- 防止ＳＱＬ注入的五种方法
- 5种方法防止 jsp被sql注入
- 【译】在React中实现条件渲染的7种方法
- winform防止界面卡死的三种方法 总结：使用委托 + 线程的方法实现 参考：https://blog.csdn.net/handsomexiaominge/article/details/4265...
- Android WifiManager.WifiLock 简介
- [译]Android内存泄漏的八种可能（上）
- python+appnium【第一章-环境搭建】
- 第一章:tornado入门环境搭建
- unittest框架核心要素及应用
- 黑马程序员前端-CSS用户界面样式源码
- 传智健康项目day01
- 传智健康项目day02
- 传智健康项目中相关知识点介绍(如图片存储,发送短信,定时调度,统计报表...)
- 黑马传智健康项目讲义第一章之ElementUI
- 黑马传智健康项目讲义第一章 之Power Designer
- 黑马传智健康项目讲义第一章之项目概述
- MYSQL的master/slave数据同步配置 ZT
评论