LIGHTON: Announcing BioClinical ModernBERT: a new state-of-the-art encoder model for Medical NLP.

DJ LIGHTON: Announcing BioClinical ModernBERT: a new state-of-the-art encoder model for Medical NLP.

LIGHTON 
LIGHTON: Announcing BioClinical ModernBERT: a new state-of-the-art encoder model for Medical NLP. 
13-Jun-2025 / 17:45 CET/CEST 
Dissemination of a French Regulatory News, transmitted by EQS Group. 
The issuer is solely responsible for the content of this announcement. 
 
=---------------------------------------------------------------------------------------------------------------------- 
  
 
  
 
Press Release 
 
Paris, June 13, 2025 
 
  
 
  
 
Announcing BioClinical ModernBERT: a new state-of-the-art encoder model for Medical NLP 
  
 
The recent release of ModernBERT by LightOn and AnswerAI aims at providing the best base model that can be then used in 
different industry verticals. 
 
  
 
Today, Thomas Sounack from the Dana-Farber Cancer Institute in collaboration with researchers at LightOn, Harvard 
University, MIT, McGill University, Albany Medical College and Microsoft Research, used this capability and trained a 
new State-Of-The-Art (SOTA) medical encoder named BioClinical ModernBERT. 
 
  
 
  
 
  
 
BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP 
 
  
 
  
 
  
 
Efficient Continued Pre-Training, Streamlined for Medicine 
 
One critical but lesser-known scheduling feature of ModernBERT allows researchers a seamless continued pre-training 
while eliminating cold restarts. Stable-phase checkpoints and a decay phase contribute to having models that can 
efficiently converge on specific domains. 
 
  
 
Leveraging this scheduling feature, Thomas Sounack continued the pre-training of ModernBERT on an extensive collection 
of medical texts. The result: BioClinical ModernBERT, a new model that outperforms all existing encoders on medical 
classification and Named Entity Recognition (NER) tasks, setting a new SOTA benchmark for medical NLP applications. 
 
  
 
Optimized for the Realities of Clinical Context 
 
Real-world medical texts can be very long; they span full clinical notes and as well as large reports. BioClinical 
ModernBERT's ModernBERT backbone provides long-context document support, with hybrid attention and unpadding mechanisms 
for rapid processing, crucial for healthcare and clinical workflows. 
 
  
 
A Recipe for Continued Pre-Training 
 
Beyond the model itself, this experience refines continued pre-training for domain adaptation. This approach is 
reproducible: BioClinical ModernBERT demonstrates robust transfer to new domains, opening the door for anyone seeking 
to tailor ModernBERT for their own specialized data. 
 
  
 
Try It Out 
 
Interested in leveraging continued pre-training or ModernBERT's long-context expertise in a different domain? Explore 
the BioClinical ModernBERT collection and see how it can advance specialized NLP tasks in your domain. 
 
  
 
About LightOn 
 
Founded in 2016 in Paris and the first European generative AI company listed on Euronext Growth, LightOn is a 
pioneering player in the field of sovereign GenAI. Its Paradigm platform enables organizations to deploy large-scale AI 
while ensuring the confidentiality of their data. LightOn's technology ensures essential strategic independence by 
offering tailored solutions. This technological mastery is accompanied by the ability to process large volumes of data 
for industrial uses, with applications in various sectors such as finance, industry, health, defense, and public 
services. 
 
LightOn is listed on Euronext Growth® Paris (ISIN: FR0013230950, ticker: ALTAI-FR). The company qualifies for PEA and 
PEA PME investment plans and is recognized as an "Innovative Company" by Bpifrance. To learn more: https:// 
www.lighton.ai  
 
Contacts 
 
                       SEITOSEI--ACTIFIN 
                       Investor Relations 
LIGHTON 
 
 
                       Benjamin LEHARI 
[e-mail] 
 
 
                       [e-mail] 
KALAMARI 
 
Media Relations               SEITOSEI--ACTIFIN 
                       Financial Media Relations 
 
 
Camille Bernisson - [phone] 
                       Jennifer JULLIA - [phone] 
 
 
Maroua Derdega - [phone] 
                       [e-mail] 
 
[e-mail] 

----------------------------------------------------------------------------------------------------------------------- Regulatory filing PDF file

File: LightOn - BioClinical ModernBERT - 061325

=---------------------------------------------------- 
Language:    English 
Company:     LIGHTON 
         2 rue de la Bourse 
         75002 Paris 
         France 
E-mail:     [e-mail] 
Internet:    www.lighton.ai 
ISIN:      FR0013230950 
Euronext Ticker: ALTAI 
AMF Category:  Inside information / Other releases 
EQS News ID:   2155084 
  
End of Announcement EQS News Service 
=------------------------------------------------------------------------------------ 

2155084 13-Jun-2025 CET/CEST

Image link: https://eqs-cockpit.com/cgi-bin/fncls.ssp?fn=show_t_gif&application_id=2155084&application_name=news&site_id=dow_jones%7e%7e%7ebed8b539-0373-42bd-8d0e-f3efeec9bbed

(END) Dow Jones Newswires

June 13, 2025 11:45 ET (15:45 GMT)