Boris Albar
I am a data scientist with a particular interest in applying deep learning to solve various problems,
from computer vision and NLU to time series analysis. I'm currenly a lead data scientist at CATIE.
In a previous life, I was interested in studying the structure of specific classes of graphs and matroids.
Portfolio
FAT5 - T5 with Flash Attention 2
FAT5 is an implementation of T5 in PyTorch with an UL2 objective optimized for GPGPU. It uses custom CUDA kernels and specific optimizations to increase throughput and reduce memory usage for both training and inference by a factor of 4. This code was used to pretrained several models in French in sizes ranging from 147M parameters to 2.5B parameters using only 2 Nvidia A100 from 300M documents from CulturaX-fr. Weights will soon be released on Hugging Face as well as adapted Flan-T5 weights.
QAmemBERT
QAmemBERT is a state-of-the-art question-answering model in French and, contrary to previously existing models in French, it support questions where the answer is not contained in the context. It is available in two sizes (base and large) and is freely available on Hugging Face. It was developed in collaboration with Loïck Bourdois and Pierre Bédu.
Triton-rs
Triton-rs is a Rust gRPC client library for NVIDIA Triton Inference Server. It also supports passing tensor through the system shared memory for fast local inference.
Few/zero shot for the French language
This conference (in French) was given for DataQuitaine 2023 and present various benchmarks for few and zero-shot learning on various French classical NLP tasks.
CTC Beam search in Rust
A fast CTC beam search decoder supporting various language models using the previously featured Triton-rs, coded in Rust. Used for fast decoding with a speech-to-text model in French.