ContinualAI Seminar: "TACOS: Task Agnostic Continual Learning in Spiking Neural Networks"
Abstract: Catastrophic interference, the loss of previously learned information when learning new information, remains a major challenge in machine learning. Since living organisms do not seem to suffer from this problem, researchers have taken inspiration from biology to improve memory retention in artificial intelligence systems. However, previous attempts to use bio-inspired mechanisms have typically resulted in systems that rely on task boundary information during training and/or explicit task identification during inference, information that is not available in real-world scenarios. Here, we show that neuro-inspired mechanisms such as synaptic consolidation and metaplasticity can mitigate catastrophic interference in a spiking neural network, using only synapse-local information, with no need for task awareness, and with a fixed memory size that does not need to be increased when training on new tasks. Our model, TACOS, combines neuromodulation with complex synaptic dynamics to enable new learning while protecting previous information. We evaluate TACOS on sequential image recognition tasks and demonstrate its effectiveness in reducing catastrophic interference. Our results show that TACOS outperforms existing regularization techniques in domain-incremental learning scenarios. We also report the results of an ablation study to elucidate the contribution of each neuro-inspired
mechanism separately.
You can discuss about this seminar in our forum: https://continualai.discourse.group/c/reading-group
------------------------ About ContinualAI ------------------------
ContinualAI is an official non-profit research organization and the largest open community on Continual Learning for AI. We aim at connecting people and working better together on this fascinating topic we consider fundamental for the future of AI.
• Official website: https://www.continualai.org
• Join us now at https://www.continualai.org/join_us
Please consider supporting us with a small donation at: https://www.continualai.org/supporters
It's thanks to people like you that we are making ContinualAI a reality!
0 Comments