Evolutionary Unconstraint Networks (EVOLUNET)

MSc project proposal 2024

Scientific field and context:

One of the main motivations of this project is to explore new neural network architectures (Neural Architecture Search) [White et al., 2023] via the use of a computational evolution algorithm. 

Despite the current predominance of approaches in which fixed, deep architectures are trained by error gradient backpropagation algorithms, the exploration of network architectures by evolutionary algorithms (gradient-free) represents a competitive and relatively unexplored alternative [Conti et al., 2018]. 

Yet the neural architecture research domain is rooted in computational evolution [Gaier and Ha, 2019], hence our motivation to rely on original evolutionary principles [Banzhaf et al., 2006]. Indeed, we want to "learn" the neural network architecture while taking advantage of the frugality of the results obtained (evolution starting from a principle of minimal complexity) and their intrinsic capabilities [Gaier and Ha, 2019], which are sometimes surprising [Lehman et al., 2020]. 

In particular, the aim is to enable the exploitation of a library of varied mutational operators, inspired by evolutionary genomics. In biology, genomes are known to be subject to a wide variety of sequence-modifying mechanisms (substitution, InDels, chromosome rearrangement, crossover, etc.) during their evolution, but "classic" evolutionary algorithms are generally based on one single mutation operator (or two a most, typically substitution and crossover). 

However, a growing number of results show that "alternative" operators (in particular rearrangements) can facilitate exploration of the search space [Trujillo et al., 2022]. Indeed, rearrangements make it possible to gene duplication/deletion, to explain the molecular complexification of organisms in the course of phylogeny. This is why we propose to integrate these operators in iteratively building a neural network through an innovative artificial evolution mechanism.

 Expected innovative contributions: 

Bioinspired mechanisms to regulate the complexity of neural networks according to the complexity of the task to be solved and the resources available to solve it.

Explore the possibility of letting the network population continue to adapt over time (Life-Long Learning), without requiring excessive external computational resources (no computational clusters, for example).

Research program and proposed scientific approach: 

Explore innovative computational bio-genomic methods to learn/individualize frugal neural network architectures.

Test chromosomal rearrangement-type mutational operators (duplication/deletion) likely to modify network complexity during evolution.

Testing simplification approaches (evolutionary pruning) to constrain network complexity [Liard et al., 2020].

Test mixed precision approaches to synaptic weights, with the evolutionary algorithm adapting both the structure of the network (via the chromosomal rearrangement mechanisms mentioned above) and the precision of the weights via "InDels" mechanisms enabling adaptation of gene length.

Compare, on simple tasks (low-dimensional classification), the low-energy neural networks obtained (intrinsic limitation of network complexity by the evolution algorithm) with state-of-the-art architectures for which complexity is predefined, generating very high consumption of computing resources.

Expected candidate profile (prerequisite): machine learning, programming. Interest for biological, bio-genomic, and/or biomedical field(s).

Skills that will be developed during the project:  strong experience in applied machine learning, integrating bio-inspired evolution mechanisms for neural network architecture design, collaborative/versionned programming. Ability to write a research article.

Contacts: 

guillaume.beslon@inria.fr, emmanuel.roux@creatis.insa-lyon.fr

White C. et al., Neural architecture search: Insights from 1000 papers, arXiv preprint arXiv:2301.08727, 2023. https://deepai.org/publication/neural-architecture-search-insights-from-1000-papers