Authors: Wissam Antoun, Fady Baly, Hazem Hajj
AraBERT is an Arabic pretrained language model based on Google’s BERT architecture. AraBERT uses the same BERT-Base config. More details are available in the AraBERT PAPER and in the AraBERT Meetup
There is two versions of the model AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the Farasa Segmenter.
The model was trained on ~70M sentences or ~23GB of Arabic text with ~3B words.
Source Code Repository: https://github.com/aub-mind/arabert
Paper: https://www.aclweb.org/anthology/2020.osact-1.2.pdf
Results (Accuracy)
We evaluate both AraBERT models on different downstream tasks and compare it to mBERT, and other state of the art models (To the extent of our knowledge). The Tasks were Sentiment Analysis on 6 different datasets (HARD, ASTD-Balanced, ArsenTD-Lev, LABR, ArSaS), Named Entity Recognition with the ANERcorp, and Arabic Question Answering on Arabic-SQuAD and ARCD
| Task | prev. SOTA | mBERT | AraBERTv0.1 | AraBERTv1 |
|---|---|---|---|---|
| HARD | 95.7 ElJundi et.al. | 95.7 | 96.2 | 96.1 |
| ASTD | 86.5 ElJundi et.al. | 80.1 | 92.2 | 92.6 |
| ArsenTD-Lev | 52.4 ElJundi et.al. | 51 | 58.9 | 59.4 |
| AJGT | 93 Dahou et.al. | 83.6 | 93.1 | 93.8 |
| LABR | 87.5 Dahou et.al. | 83 | 85.9 | 86.7 |
| ANERcorp | 81.7 (BiLSTM-CRF) | 78.4 | 84.2 | 81.9 |
| ARCD | mBERT | EM:34.2 F1: 61.3 | EM:51.14 F1:82.13 | EM:54.84 F1: 82.15 |
Model Weights and Vocab Download
| Models | AraBERTv0.1 | AraBERTv1 |
|---|---|---|
| TensorFlow | Drive Link | Drive Link |
| PyTorch | Drive_Link | Drive_Link |
You can find the PyTorch models in HuggingFace’s Transformer Library under the aubmindlab username
If you used this model please cite us as:
@inproceedings{antoun2020arabert,
title={AraBERT: Transformer-based Model for Arabic Language Understanding},
author={Antoun, Wissam and Baly, Fady and Hajj, Hazem},
booktitle={LREC 2020 Workshop Language Resources and Evaluation Conference 11--16 May 2020},
pages={9}
}
Acknowledgments
Thanks to TensorFlow Research Cloud (TFRC) for the free access to Cloud TPUs, couldn’t have done it without this program, and to the AUB MIND Lab Members for the continous support. Also thanks to Yakshof and Assafir for data and storage access. Another thanks for Habib Rahal (https://www.behance.net/rahalhabib), for putting a face to AraBERT.
Contacts
Wissam Antoun: Linkedin | Twitter | Github | wfa07@mail.aub.edu | wissam.antoun@gmail.com
Fady Baly: Linkedin | Twitter | Github | fgb06@mail.aub.edu | baly.fady@gmail.com
We are looking for sponsors to train BERT-Large and other Transformer models, the sponsor only needs to cover to data storage and compute cost of the generating the pretraining data
Experience the best IPTV Dubai service with King IPTV Dubai. Get access to 10,000+ live TV channels and 12,000+ movies & series in stunning HD, 4K, and 10K quality. Our advanced anti-freeze technology ensures smooth and buffer-free streaming for users in Dubai, Abu Dhabi, Sharjah, and across the UAE. Enjoy instant activation and 24/7 customer support for an unmatched entertainment experience.
Truly, this article is one of the very best in the history of articles. I am an antique ’Article’ collector and I sometimes read some new articles if I find them interesting. And I found this one pretty fascinating and it should go into my collection. Very good work. party busses near me
King IPTV Dubai – The #1 IPTV Service in UAE
Looking for the best IPTV Dubai service? King IPTV Dubai offers premium IPTV solutions with 10,000+ live channels and 12,000+ movies & series in HD, 4K, and even 10K quality. Enjoy buffer-free streaming with our advanced anti-freeze technology, ensuring a seamless viewing experience. With instant activation and 24/7 customer support, we cater to users in Dubai, Abu Dhabi, Sharjah, and across the UAE. Whether you’re into sports, entertainment, or international content, King IPTV Dubai provides a top-tier streaming experience.
Get the best IPTV service in Dubai today! Visit kingiptvdubai.com.
Experience the best IPTV Dubai service with King IPTV Dubai. Get access to 10,000+ live TV channels and 12,000+ movies & series in stunning HD, 4K, and 10K quality. Our advanced anti-freeze technology ensures smooth and buffer-free streaming for users in Dubai, Abu Dhabi, Sharjah, and across the UAE. Enjoy instant activation and 24/7 customer support for an unmatched entertainment experience.
Experience the best IPTV Dubai service with King IPTV Dubai. Get access to 10,000+ live TV channels and 12,000+ movies & series in stunning HD, 4K, and 10K quality. Our advanced anti-freeze technology ensures smooth and buffer-free streaming for users in Dubai, Abu Dhabi, Sharjah, and across the UAE. Enjoy instant activation and 24/7 customer support for an unmatched entertainment experience.
Experience the best IPTV Dubai service with King IPTV Dubai. Get access to 10,000+ live TV channels and 12,000+ movies & series in stunning HD, 4K, and 10K quality. Our advanced anti-freeze technology ensures smooth and buffer-free streaming for users in Dubai, Abu Dhabi, Sharjah, and across the UAE. Enjoy instant activation and 24/7 customer support for an unmatched entertainment experience.
Data SDY Tercepat mengacu pada hasil pengeluaran Sydney (SDY) yang diperbarui secara real-time, memungkinkan para pemain togel untuk mendapatkan informasi terbaru dengan akurat dan cepat. Dengan akses ke data SDY tercepat, pemain dapat menganalisis angka-angka sebelumnya untuk menyusun strategi permainan yang lebih baik.
Data SDY Tercepat mengacu pada hasil pengeluaran Sydney (SDY) yang diperbarui secara real-time, memungkinkan para pemain togel untuk mendapatkan informasi terbaru dengan akurat dan cepat. Dengan akses ke data SDY tercepat, pemain dapat menganalisis angka-angka sebelumnya untuk menyusun strategi permainan yang lebih baik.
VZ99 hiện đang là một trong những nhà cái hàng đầu châu Á, nổi bật với hệ thống cá cược trực tuyến hợp pháp, uy tín và an toàn. Với sứ mệnh mang đến cho người chơi những trải nghiệm đỉnh cao, VZ99 không chỉ cung cấp các trò chơi cá cược thể thao hấp dẫn mà còn có một loạt các dịch vụ trực tuyến, game slot, và các loại hình giải trí khác.
VZ99 hiện đang là một trong những nhà cái hàng đầu châu Á, nổi bật với hệ thống cá cược trực tuyến hợp pháp, uy tín và an toàn. Với sứ mệnh mang đến cho người chơi những trải nghiệm đỉnh cao, VZ99 không chỉ cung cấp các trò chơi cá cược thể thao hấp dẫn mà còn có một loạt các dịch vụ trực tuyến, game slot, và các loại hình giải trí khác.
With so many books and articles coming up to give gateway to make-money-online field and confusing reader even more on the actual way of earning money, health news
Chúng ta thường đồng nhất hành vi lạm dụng trẻ em với thuật ngữ ấu dâm dùng để nói đến tất cả người có bất cứ chủ đề tình dục nào với trẻ em. #audam #hanhviaudam
Non-emergency medical transportation rates vary based on distance, vehicle type, and required assistance. Wheelchair van trips typically cost $50–$150, while stretcher transport ranges from $100–$500+. Non-emergency ambulance services can exceed $200. Prices depend on location, provider, and insurance coverage, with some costs covered by Medicaid or private insurance.
https://usbargainlimo.com/party-bus-philadelphia/
This post is very simple to read and appreciate without leaving any details out. Great work! wheon subway surfers for pc
It has fully emerged to crown Singapore’s southern shores and undoubtedly placed her on the global map of residential landmarks. I still scored the more points than I ever have in a season for GS. I think you would be hard pressed to find somebody with the same consistency I have had over the years so I am happy with that. sri lanka national cricket team vs india national cricket team timeline
J88 là nhà cái cá cược trực tuyến uy tín top #1 thị trường Châu Á, sở hữu kho game đổi thưởng đồ sộ từ nổ hũ, bắn cá, thế thao đến đá gà, mang đến cho người chơi những trải nghiệm giải trí đỉnh cao đầy thú vị. J 88 cam kết rõ ràng, minh bạch và tính bảo mật an toàn luôn được đặt lên hàng đầu, giúp thương hiệu xây dựng niềm tin vững chắc trong cộng đồng game thủ.
Kubet la 1 nha cai giai tri online uy tin tai thi truong Viet Nam. Nha cai cung cap cac tro choi da dang nhu the thao, game bai, no hu, ban ca va nhieu game hot khac. Kubet cam ket mang den cho nguoi choi nhung nghiem an toan, minh bach va cong bang, voi giao dien de su dung va khuyen mai hap dan!
Link truy cap an toan: https://1bkubet.com/
We are tied directly into the sate’s renewal database which allows us to process your request almost instantly. remove dark spots on face tang – lemon juice
123B là một nền tảng giải trí trực tuyến hàng đầu, mang đến nhiều sản phẩm hấp dẫn và dịch vụ chất lượng cao. Với sự đa dạng trong nội dung, 123B không ngừng cập nhật và cải tiến để đáp ứng nhu cầu của người dùng, mang đến những trải nghiệm thú vị và đáng tin cậy. 123B
Yacine TV has emerged as a leading platform for sports enthusiasts seeking free access to live streaming of matches and major sporting events from around the globe. This application stands out in the crowded space of sports streaming by offering high-quality broadcasts without the hassle of subscriptions or hidden fees. Designed to cater to fans of both Arab and international leagues yacine tv ensures that users never miss a moment of their favorite games whether they follow local clubs or top-tier international competitions. The app’s commitment to providing seamless real-time coverage has made it a go-to choice for millions looking to stay connected to the sports they love.
It is one of my favourite blog, I often come to read here. Keep it up giftcardmall.com mygift
Slot88 is a favorite choice for many players because of its fun and simple gameplay. The platform offers different slot games with great graphics and exciting features. Slot88 is easy to access and perfect for both new and experienced players. If you love online gaming, Slot88 is a good choice for entertainment.
Through this post, I know that your good knowledge in playing with all the pieces was very helpful. I notify that this is the first place where I find issues I’ve been searching for. You have a clever yet attractive way of writing. interesting news hearthstats
Эксперты компании UA-OFFSHORE помогут открыть фирму в Гонконге за короткий промежуток времени и со 100% гарантией успеха. За годы работы мы накопили достаточно опыта, позволяющего профессионально выполнять все поставленные перед нами задачи. Мы прекрасно разбираемся во всех нюансах законодательной и налоговой базы Гонконга, поэтому беспроблемно пройдем все этапы создания фирмы.
Зарегистрировать Офшор в Гонконге
With so many books and articles coming up to give gateway to make-money-online field and confusing reader even more on the actual way of earning money, Harmonicode
This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea. upcoming fixtures sffareboxing
I have express a few of the articles on your website now, and I really like your style of blogging. I added it to my favorite’s blog site list and will be checking back soon… how to get on first page of google search david aziz
With so many books and articles coming up to give gateway to make-money-online field and confusing reader even more on the actual way of earning money, AbitHelp Tablets
With so many books and articles coming up to give gateway to make-money-online field and confusing reader even more on the actual way of earning money, mommyandlove
This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea. TechDae
The next time I read a blog, I hope that it doesnt disappoint me as much as this one. I mean, I know it was my choice to read, but I actually thought you have something interesting to say. All I hear is a bunch of whining about something that you could fix if you werent too busy looking for attention. millionaire life
Kuwin là nền tảng giải trí trực tuyến hàng đầu, mang đến cho người dùng trải nghiệm đa dạng và phong phú. Với sự đầu tư vào công nghệ tiên tiến, Kuwin đảm bảo mang lại môi trường an toàn, minh bạch và chuyên nghiệp. Giao diện trang web được thiết kế trực quan, giúp người dùng dễ dàng thao tác và tận hưởng những dịch vụ hấp dẫn. Kuwin
majnulaila.com is the most well-known call girl agency in Dehradun. vip call girls in Dehradun In Dehradun, our agency provides stunning and intelligent call ladies. Our models are attractive and seductive. Our call girls are offered at incredibly low costs. We offer both after-hours and in-call services to our clients. Our company’s primary goal is to deliver the best possible client service. People like us because of this. Our service covers the entire city of Dehradun.
Với tầm nhìn phát triển bền vững, Kuwin không ngừng mở rộng và hoàn thiện hệ sinh thái giải trí của mình. Cam kết mang đến chất lượng dịch vụ hàng đầu, Kuwin đã và đang trở thành sự lựa chọn tin cậy của đông đảo người dùng. Kuwin
In the ever-evolving landscape of digital media, a new player has emerged that’s causing quite a stir: Erome Media. Whether you’re a content creator, a consumer of digital entertainment, or someone curious about the latest trends in online platforms, Erome Media is a name you should become familiar with. But what exactly is Erome Media, and why should it be on your radar?
A leather jacket with a leather hood is the perfect fusion of rugged durability and sleek fashion. This design not only enhances warmth and protection but also adds an urban edge to any outfit. Whether styled casually with jeans or layered for a more refined look, it offers versatility and timeless appeal. Ideal for those who appreciate bold yet practical outerwear, this jacket ensures both comfort and sophistication, making it a staple piece for any wardrobe.
Nulls Brawl is a private server for Brawl Stars with unlimited resources. Play with all brawlers, skins, and custom mods without restrictions. Enjoy fast-paced battles with unlimited gems and coins!
SV66 tự hào là nền tảng giải trí hàng đầu Châu Á năm 2025, mang đến cho người dùng trải nghiệm đẳng cấp với nhiều lựa chọn hấp dẫn. Với cam kết không ngừng nâng cao chất lượng dịch vụ, SV66 mang đến không gian giải trí phong phú, phù hợp với mọi sở thích và nhu cầu. SV66
Không chỉ chú trọng đến sự đa dạng, SV66 còn đầu tư mạnh mẽ vào công nghệ để mang đến hệ thống bảo mật hàng đầu. Tất cả thông tin và giao dịch đều được mã hóa chặt chẽ, đảm bảo sự an toàn tuyệt đối cho người dùng. Đây chính là yếu tố quan trọng giúp SV66 trở thành nền tảng giải trí đáng tin cậy. SV66
Thanks for sharing the post.. parents are worlds best person in each lives of individual..they need or must succeed to sustain needs of the family. 8kbet vip
QQ88 là nền tảng giải trí trực tuyến hàng đầu tại Việt Nam. Dù mới ra mắt nhưng nó nổi bật với các sản phẩm, dịch vụ được đánh giá cao như trò chơi tương tác, thử thách kỹ năng, thể thao, thể thao điện tử, v.v. Với cam kết đặt khách hàng lên hàng đầu, QQ88 sẽ tiếp tục phát huy thế mạnh và cải thiện trải nghiệm người dùng, đảm bảo một nền tảng hấp dẫn và thú vị hơn cho tất cả mọi người. QQ88