Authors: Wissam Antoun, Fady Baly, Hazem Hajj
AraBERT is an Arabic pretrained language model based on Google’s BERT architecture. AraBERT uses the same BERT-Base config. More details are available in the AraBERT PAPER and in the AraBERT Meetup
There is two versions of the model AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the Farasa Segmenter.
The model was trained on ~70M sentences or ~23GB of Arabic text with ~3B words.
Source Code Repository: https://github.com/aub-mind/arabert
Paper: https://www.aclweb.org/anthology/2020.osact-1.2.pdf
Results (Accuracy)
We evaluate both AraBERT models on different downstream tasks and compare it to mBERT, and other state of the art models (To the extent of our knowledge). The Tasks were Sentiment Analysis on 6 different datasets (HARD, ASTD-Balanced, ArsenTD-Lev, LABR, ArSaS), Named Entity Recognition with the ANERcorp, and Arabic Question Answering on Arabic-SQuAD and ARCD
| Task | prev. SOTA | mBERT | AraBERTv0.1 | AraBERTv1 |
|---|---|---|---|---|
| HARD | 95.7 ElJundi et.al. | 95.7 | 96.2 | 96.1 |
| ASTD | 86.5 ElJundi et.al. | 80.1 | 92.2 | 92.6 |
| ArsenTD-Lev | 52.4 ElJundi et.al. | 51 | 58.9 | 59.4 |
| AJGT | 93 Dahou et.al. | 83.6 | 93.1 | 93.8 |
| LABR | 87.5 Dahou et.al. | 83 | 85.9 | 86.7 |
| ANERcorp | 81.7 (BiLSTM-CRF) | 78.4 | 84.2 | 81.9 |
| ARCD | mBERT | EM:34.2 F1: 61.3 | EM:51.14 F1:82.13 | EM:54.84 F1: 82.15 |
Model Weights and Vocab Download
| Models | AraBERTv0.1 | AraBERTv1 |
|---|---|---|
| TensorFlow | Drive Link | Drive Link |
| PyTorch | Drive_Link | Drive_Link |
You can find the PyTorch models in HuggingFace’s Transformer Library under the aubmindlab username
If you used this model please cite us as:
@inproceedings{antoun2020arabert,
title={AraBERT: Transformer-based Model for Arabic Language Understanding},
author={Antoun, Wissam and Baly, Fady and Hajj, Hazem},
booktitle={LREC 2020 Workshop Language Resources and Evaluation Conference 11--16 May 2020},
pages={9}
}
Acknowledgments
Thanks to TensorFlow Research Cloud (TFRC) for the free access to Cloud TPUs, couldn’t have done it without this program, and to the AUB MIND Lab Members for the continous support. Also thanks to Yakshof and Assafir for data and storage access. Another thanks for Habib Rahal (https://www.behance.net/rahalhabib), for putting a face to AraBERT.
Contacts
Wissam Antoun: Linkedin | Twitter | Github | wfa07@mail.aub.edu | wissam.antoun@gmail.com
Fady Baly: Linkedin | Twitter | Github | fgb06@mail.aub.edu | baly.fady@gmail.com
We are looking for sponsors to train BERT-Large and other Transformer models, the sponsor only needs to cover to data storage and compute cost of the generating the pretraining data
Welcome to Prime Path—your ultimate solution for starting and growing a profitable Amazon store without the hassle. We offer all-done-for-you Amazon stores, making it easier than ever to step into the world of e-commerce. Amazon store
I have bookmarked your website because this site contains valuable information in it. I am really happy with articles quality and presentation. Thanks a lot for keeping great stuff. I am very much thankful for this site. crypto30x.com asx
Anyone looking for quick access to Apple services might consider aniCloud account for sale. It’s useful for those who need extra storage or want a new account without setting up from scratch.
Ajaib88 offers a variety of exciting games for you, from online slots to classic card games. All are designed to provide the best entertainment and great chances to win.
The next time I read a blog, I hope that it doesnt disappoint me as much as this one. I mean, I know it was my choice to read, but I actually thought you have something interesting to say. All I hear is a bunch of whining about something that you could fix if you werent too busy looking for attention. Pedrovazpaulo Business Consultant
Slot88 merupakan hoki slot Asia Tergacor paling viral dan sudah terbukti sering memberikan maxwin dan jp kepada para playar, hanya modal receh dan main di bet 200 perak kalian bisa meraup keuntungan jutaan rupiah setiap harinya Slot hoki
I am a new user of this site so here i saw multiple articles and posts posted by this site,I curious more interest in some of them hope you will give more information on this topics in your next articles. Abraham Quiros Villalba
Nice to read your article! I am looking forward to sharing your adventures and experiences. agenolx
Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon. Big thanks for the useful info. cane corso for sale near me
I have bookmarked your website because this site contains valuable information in it. I am really happy with articles quality and presentation. Thanks a lot for keeping great stuff. I am very much thankful for this site. chosenviber.net
The Paragraph Summarizer tool is incredibly helpful for quickly extracting the key points from lengthy texts Paragraph Summarizer
Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. create budget
I am definitely bookmarking this website and sharing it with my acquaintances. You will be getting plenty of visitors to your website from me! slot gacor hari ini
Thank you because you have been willing to share information with us. We will always appreciate all you have done here because I know you are very concerned with us. ghg automation system
This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea. rs 125 only on thesparkshop.in batman style wireless bt earbuds
Hi there to every one, since I am really keen of reading this weblog’s
post to be updated daily. It carries pleasant data.
Also visit my web site – play best games online
The essay title generator is a creative and helpful tool for students, writers, and professionals looking for inspiration. essay title generator
The essay title generator is a creative and helpful tool for students, writers, and professionals looking for inspiration. essay title generator
I’m not that much of a online reader to be honest but your blogs really nice, keep
it up! I’ll go ahead and bookmark your site to come back later
on. Cheers
Here is my blog post: next page
This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea. get_ready_bell:client_pulse
I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page! phoenix suns vs la clippers match player stats
This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea. PI123
First Choice Plumbing Heating & Air Conditioning is absolutely amazing! Their team is professional, punctual, and always delivers top-quality work. I recently used their ac service in Wayne NJ, and they exceeded my expectations. My cooling system is running perfectly now, and they even provided helpful maintenance tips. Highly recommend them for anyone looking for reliable and efficient services!
Quickly this particular site may irrefutably gain popularity in between just about all running a blog as well as site-building people, in order to it’s diligent articles or maybe evaluations. RTP Rakyatjp
Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. knicks vs pacers match player stats
I’m not sure why but this website is loading very slow for me.
Is anyone else having this issue or is it a problem on my end?
I’ll check back later and see if the problem still exists.
Feel free to visit my webpage – kra23 cc
Top Egypt Tour Packages & Vacation Deals 2025, Best Egypt Vacation Packages 2025 – Special Prices Hurghada, marsa alam, safaga , elgouna, sahl Hasheesh, Makadi
Book Hurghada excursions
A book title generator is a great tool for sparking creativity and finding inspiration when coming up with the perfect title for a book. book title generator
Detect AI-generated content instantly with our free AI Content Detector tool for ChatGpt, deepseek, perplexity and more. Analyze text from PDF, DOC, TXT, and more to identify AI-generated patterns and ai-generated text. Fast, accurate, and easy to use!
detecteur ia
Nice information, valuable and excellent design, as share good stuff with good ideas and concepts, lots of great information and inspiration, both of which I need, thanks to offer such a helpful information here. Save Money
Nice information, valuable and excellent design, as share good stuff with good ideas and concepts, lots of great information and inspiration, both of which I need, thanks to offer such a helpful information here. Save Money
This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea. Increditools
Quickly this site might undoubtedly become well-known among most blogging and site-building people, for that painstaking posts as well as critiques. RTP Rakyatjp
MoviePlus es un software de edición de vídeo fácil de usar que permite crear y editar vídeos fácilmente.Ofrece diversas herramientas efectos y plantillas por lo que es perfecto para que principiantes y usuarios experimentados mejoren sus proyectos de vídeo.movieplus online
With so many books and articles coming up to give gateway to make-money-online field and confusing reader even more on the actual way of earning money, Drew Brees Makes His NBC Debut, Internet Amazed by His New Hair
This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea. how to get on first page of google search david aziz
Jak navštívit Egypt? Bez ohledu na to, odkud pocházíte, Egypt povoluje vstupní nebo turistická víza všem. Návštěva Egypta se stala neuvěřitelně výhodnou s předložením elektronického vízového online procesu. Tento oficiální cestovní záznam, jinak nazývaný evisa, umožňuje cestujícímu požádat o vízum a získat je na webu. Nyní nemusíte navštěvovat egyptskou ambasádu. Co je to e-vízum do Egypta? Egyptské e-vízum je elektronické povolení k cestování, které obdržíte e-mailem. Žádost o vízum do Egypta může být podána z jakékoli země na planetě, z vašeho domova, kanceláře nebo v každém případě, když spěcháte. Celý proces žádosti o vízum je kompletně online, včetně schválení a platby eVisa. Vízová legitimita Elektronické vízum do Egypta se uděluje cestujícím, kteří hledají krátkodobý pobyt. Legitimita návštěvy víza je třicet dní nebo jeden měsíc. O egyptské e-vízum můžete požádat na webu a podporované vízum ukázat v elektronickém nebo tištěném duplikátu pohraničním imigračním úředníkům. Jak byste mohli požádat o e-vízum pro vstup do Egypta? Cyklus je velmi přímočarý a jednoduchý. eta egypt
This blog post is a gem! Thank you for sharing such valuable information. If you’re interested in diving deeper into similar topics, make sure to visit Packaging printing for more enlightening reads
Kudos to the author for crafting such a well-researched and informative blog post. If you’re as intrigued by this topic as I am, I encourage you to visit Magnet for even more valuable insights.
I’m impressed by the thoroughness of this blog post. It’s evident that a lot of time and effort went into its creation. If you’re hungry for more fascinating content, Money packet offers a plethora of resources waiting to be explored.
“如果你对冰块冻结的时间和原因感兴趣，我在网上找到了一篇非常有用的文章，提供了详细的解释和信息。你可以通过 点击这里 了解更多关于冰块冷冻时间的知识。希望对你有所帮助！”
I am glad that it turned out so well and I hope it will continue in the future because it is so worthwhile and meaningful to the community. slot bonus 100
The shortened paragraph effectively retains the main ideas and improves readability. However, it’s important to ensure no critical details are lost during the process. paragraph shortener
Excellent blog! Thank you very much for sharing this topic’s valuable information. This platform is refreshing since it provides readers with interesting and thoroughly researched information. I would like to draw attention to the fact that our platforms now provide possibilities for those who have a flare for writing and are interested about trends to Write for us technology. This might be a great opportunity to meet other computer enthusiasts who share your interests, develop your online profile, and exchange expertise. If you’re seeking new viewpoints or contributors to help you grow your content, asking individuals to Write for us Technology might help your platform attract a variety of creative and original ideas. Once again, thank you for the insightful article. I hope to read more from you!
For most up-to-date information you have to pay a quick visit web and on world-wide-web I found this website as a finest website
for hottest updates.
Here is my page :: More about the author
GALuft is your professional and reliable partner for rental, maintenance and sales of air purifiers, dehumidifiers, construction dryers, asbestos vacuum cleaners and measuring technology. Our offer is aimed at commercial, industrial and trade sectors in order to meet a wide range of requirements efficiently and reliably. bau-luftreiniger
Quickly this particular excellent website will definitely irrefutably continually be famous among the majority of composing the weblog women and men, simply because persistent content material in addition to recommendations. LOTTERY DEFEATER
so… facebook recommended me the pages “food” and “eating” ,,, yeah I’m obese|HasmAttack| ثبت نام جت بت
The rewritten sentence captures the essence of the original while improving clarity and flow. sentence rewriter
Through innovative solutions in metal additive manufacturing, FormAlloy is revolutionizing how industries design and produce metal parts, offering unmatched efficiency, flexibility, and sustainability.