Authors: Wissam Antoun, Fady Baly, Hazem Hajj
AraBERT is an Arabic pretrained language model based on Google’s BERT architecture. AraBERT uses the same BERT-Base config. More details are available in the AraBERT PAPER and in the AraBERT Meetup
There is two versions of the model AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the Farasa Segmenter.
The model was trained on ~70M sentences or ~23GB of Arabic text with ~3B words.
Source Code Repository: https://github.com/aub-mind/arabert
Paper: https://www.aclweb.org/anthology/2020.osact-1.2.pdf
Results (Accuracy)
We evaluate both AraBERT models on different downstream tasks and compare it to mBERT, and other state of the art models (To the extent of our knowledge). The Tasks were Sentiment Analysis on 6 different datasets (HARD, ASTD-Balanced, ArsenTD-Lev, LABR, ArSaS), Named Entity Recognition with the ANERcorp, and Arabic Question Answering on Arabic-SQuAD and ARCD
| Task | prev. SOTA | mBERT | AraBERTv0.1 | AraBERTv1 |
|---|---|---|---|---|
| HARD | 95.7 ElJundi et.al. | 95.7 | 96.2 | 96.1 |
| ASTD | 86.5 ElJundi et.al. | 80.1 | 92.2 | 92.6 |
| ArsenTD-Lev | 52.4 ElJundi et.al. | 51 | 58.9 | 59.4 |
| AJGT | 93 Dahou et.al. | 83.6 | 93.1 | 93.8 |
| LABR | 87.5 Dahou et.al. | 83 | 85.9 | 86.7 |
| ANERcorp | 81.7 (BiLSTM-CRF) | 78.4 | 84.2 | 81.9 |
| ARCD | mBERT | EM:34.2 F1: 61.3 | EM:51.14 F1:82.13 | EM:54.84 F1: 82.15 |
Model Weights and Vocab Download
| Models | AraBERTv0.1 | AraBERTv1 |
|---|---|---|
| TensorFlow | Drive Link | Drive Link |
| PyTorch | Drive_Link | Drive_Link |
You can find the PyTorch models in HuggingFace’s Transformer Library under the aubmindlab username
If you used this model please cite us as:
@inproceedings{antoun2020arabert,
title={AraBERT: Transformer-based Model for Arabic Language Understanding},
author={Antoun, Wissam and Baly, Fady and Hajj, Hazem},
booktitle={LREC 2020 Workshop Language Resources and Evaluation Conference 11--16 May 2020},
pages={9}
}
Acknowledgments
Thanks to TensorFlow Research Cloud (TFRC) for the free access to Cloud TPUs, couldn’t have done it without this program, and to the AUB MIND Lab Members for the continous support. Also thanks to Yakshof and Assafir for data and storage access. Another thanks for Habib Rahal (https://www.behance.net/rahalhabib), for putting a face to AraBERT.
Contacts
Wissam Antoun: Linkedin | Twitter | Github | wfa07@mail.aub.edu | wissam.antoun@gmail.com
Fady Baly: Linkedin | Twitter | Github | fgb06@mail.aub.edu | baly.fady@gmail.com
We are looking for sponsors to train BERT-Large and other Transformer models, the sponsor only needs to cover to data storage and compute cost of the generating the pretraining data
In their normal site, you’ll see that internet site, it is advisable to know of which truth. mpo888
This is very realistic, though might be vital to guideline mass media which web page webpage website page web page link: mpo888
In such a internet site you could fully grasp this likes and dislikes, crank out whatever special. mpo888
Thanks for sharing this valuable information! If you’re looking for top-notch transportation, Sarna Cab Services offers premium Cab Services in Ranchi, catering to both daily commuters and travelers. Our commitment to safety, comfort, and punctuality makes us the preferred choice for all travel needs in the city.
23win is a highly rated platform in online entertainment today. With 10 years of working experience, the playground has brought many achievements. Please refer now at: https://23win.kim/
Saya suka bagaimana Anda merangkum informasi kompleks menjadi sesuatu yang mudah dipahami. DURABELT EP 125 Rubber Conveyor Belts 4 Ply BW 600
I have bookmarked your website because this site contains valuable information in it. I am really happy with articles quality and presentation. Thanks a lot for keeping great stuff. I am very much thankful for this site. icryptoai
RajaCabe88 is a fantastic gaming site with a wide variety of games and a seamless experience!.
Link here RAJACABE88
Pusatliga88 merupakan situs slot gacor hari ini yang paling populer di tahun 2025 serta situs togel online terbaik sejak tahun 2015. Pusatliga88 Login juga menyediakan banyak game seperti Slot kakek zeus, Togel hongkong, game mahyong ways dan masih banyak lagi. Pusatliga88
Saya suka bagaimana Anda merangkum informasi kompleks menjadi sesuatu yang mudah dipahami. DURABELT EP 125 Rubber Conveyor Belts 4 Ply BW 600
Lixi88 is a platform that has gained recognition for its diverse offerings and user-friendly experience. With a well-designed interface, it caters to a broad audience looking for engaging entertainment options. The platform emphasizes smooth navigation and accessibility, ensuring that users can easily find what they need without unnecessary complications. Whether accessing the platform via desktop or mobile devices, Lixi88 provides a seamless experience that keeps users engaged. Lixi88
Introducing a selection of common sense and knowledge that you’ll want to share with someone, as well as fun trivia and trivia that you can use in everyday conversation. Know More Here: 常識
Artikel yang sangat inspiratif! Saya merasa lebih termotivasi sekarang. DURABELT EP 125 Rubber Conveyor Belts 4 Ply BW 800
Wow, excellent post. I’d like to draft like this too – taking time and real hard work to make a great article. This post has encouraged me to write some posts that I am going to write soon. long term car leasing qatar
I have bookmarked your website because this site contains valuable information in it. I am really happy with articles quality and presentation. Thanks a lot for keeping great stuff. I am very much thankful for this site. Augusta Precious Metals Lawsuit
Nohu là nhà cái cá cược hàng đầu Việt Nam với nhiều sản phẩm cá cược chất lượng như thể thao, game bài, nổ hũ,… Truy cập ngay nohu79.online nhận ngay giftcode 999k trải nghiệm.
– GG Site: https://sites.google.com/view/nohu79online/
Artikel yang sangat inspiratif! Saya merasa lebih termotivasi sekarang. DURABELT EP 125 Rubber Conveyor Belts 4 Ply BW 800
Với nhiều chương trình khuyến mãi hấp dẫn, 789Club luôn mang đến cơ hội khai thác
Artikel yang sangat inspiratif! Saya merasa lebih termotivasi sekarang. DURABELT EP 125 Rubber Conveyor Belts 4 Ply BW 800
One of the standout aspects of Lixi88 is its dedication to customer satisfaction. The platform prioritizes user feedback, constantly working to enhance features and services based on the needs of its audience. Regular updates and improvements ensure that users always have access to the best possible experience. Additionally, Lixi88 maintains a strong commitment to security, implementing advanced measures to protect user data and provide a safe environment for all its members. Lixi88
I have bookmarked your website because this site contains valuable information in it. I am really happy with articles quality and presentation. Thanks a lot for keeping great stuff. I am very much thankful for this site. Qiuzziz
sailor moon jackets Needed to draft you the bit of note to be able to thank you again just for the breathtaking pointers you’ve shared at this time. It is certainly incredibly open-handed with people like you to supply easily precisely what a lot of folks would’ve offered as an e book to help make some bucks on their own, notably now that you could have done it in case you decided. The tricks additionally served to become good way to recognize that some people have a similar interest the same as my own to learn many more concerning this problem. I am sure there are a lot more enjoyable periods ahead for many who scan through your blog.
Slot gacor is a popular term among online slot players. It means a slot game that gives high payouts and frequent wins. Many players look for slot gacor to increase their chances of winning. Some believe certain times or strategies can make a slot gacor. However, luck plays a big role. Always play responsibly and enjoy the game for fun, not just for money.
Artikel yang sangat inspiratif! Saya merasa lebih termotivasi sekarang. DURABELT EP 125 Rubber Conveyor Belts 4 Ply BW 800
789WIN is the most prominent brand in the online entertainment industry to date, the lobby has attracted more than 50 million members. Refer to the website: https://r789win.com/
Pusatliga88 merupakan situs slot gacor hari ini yang paling populer di tahun 2025 serta situs togel online terbaik sejak tahun 2015. Pusatliga88 Login juga menyediakan banyak game seperti Slot kakek zeus, Togel hongkong, game mahyong ways dan masih banyak lagi. Slot Gacor Hari Ini
A key element of Lixi88’s success lies in its ability to adapt to technological advancements. By staying ahead of trends, the platform ensures that users have access to the latest innovations, making their experience more enjoyable and efficient. The integration of cutting-edge technology allows for smoother performance, enhancing the overall usability of the platform. Lixi88
A key element of Lixi88’s success lies in its ability to adapt to technological advancements. By staying ahead of trends, the platform ensures that users have access to the latest innovations, making their experience more enjoyable and efficient. The integration of cutting-edge technology allows for smoother performance, enhancing the overall usability of the platform. Lixi88
A key element of Lixi88’s success lies in its ability to adapt to technological advancements. By staying ahead of trends, the platform ensures that users have access to the latest innovations, making their experience more enjoyable and efficient. The integration of cutting-edge technology allows for smoother performance, enhancing the overall usability of the platform. Lixi88
Thank you for sharing such an informative blog! If you’re looking to Study MBBS in Russia, Medjakk provides expert guidance and assistance to help students secure admissions in top Russian medical universities. With affordable fees, globally recognized degrees, and high-quality education, Russia is a preferred destination for medical aspirants. Visit our website for more details on MBBS admissions and make your dream of becoming a doctor a reality!
We are tied directly into the sate’s renewal database which allows us to process your request almost instantly. crypto30x.com ac milan
I have bookmarked your website because this site contains valuable information in it. I am really happy with articles quality and presentation. Thanks a lot for keeping great stuff. I am very much thankful for this site. LogicalShout News:
kubettt3.com không chỉ khẳng định vị thế là một nền tảng cá cược trực tuyến chuyên nghiệp mà còn cam kết mang đến sự an toàn tuyệt đối cho người chơi. Với hệ thống bảo mật tiên tiến, KUBET đảm bảo mọi thông tin cá nhân và giao dịch của người dùng luôn được bảo vệ nghiêm ngặt. Chính sự uy tín, minh bạch và dịch vụ chất lượng đã giúp KUBET trở thành lựa chọn hàng đầu của đông đảo người chơi.
Thanks for the nice blog. It was very useful for me. I m happy I found this blog. Thank you for sharing with us,I too always learn something new from your post. bigtechoro
I’m happy I located this blog! From time to time, students want to cognitive the keys of productive literary essays composing. Your first-class knowledge about this good post can become a proper basis for such people. nice one 5starsstocks.com staples
I’m happy I located this blog! From time to time, students want to cognitive the keys of productive literary essays composing. Your first-class knowledge about this good post can become a proper basis for such people. nice one ecryptobit.com nft
Terima kasih atas pemikiran yang dalam dan terperinci. Saya sangat menghargainya. DURABELT EP 125 Rubber Conveyor Belts 5 Ply BW 500
J88 will be the ideal online entertainment location when you want to find a place to relieve stress after work. Come to us https://j88casino.info/
Fitur-fitur di nadia4d slot cukup lengkap, dari mode multiplayer hingga berbagai tantangan menarik yang membuat permainan semakin seru.
79sodo offers a seamless and interactive entertainment experience with a variety of engaging activities, ensuring users stay entertained at all times. 79sodo
Security measures are in place to protect user information, ensuring that personal details and transactions remain safe from unauthorized access. 79sodo
nadia4d rtp selalu menghadirkan pembaruan dan event terbaru, sehingga pengalaman bermain tidak membosankan dan selalu ada hal baru untuk dicoba.
Frequent updates and new additions keep the experience fresh, preventing monotony and encouraging long-term engagement. 79sodo
Terima kasih atas pemikiran yang dalam dan terperinci. Saya sangat menghargainya. DURABELT EP 125 Rubber Conveyor Belts 5 Ply BW 500
Nohu là cổng game xanh chín nhất hiện tại trên thị trường cá cược Việt Nam. Với chính sách bảo mật và tính minh bạch cực cao, Nohu đang thu hút rất nhiều anh em tham gia trải nghiệm.
GG Site: https://sites.google.com/view/nohu79app/
Sangat menyukai pendekatan Anda dalam menangani topik ini. Sangat terperinci dan informatif! DURABELT EP 125 Rubber Conveyor Belts 5 Ply BW 600
Thanks for the nice blog. It was very useful for me. I m happy I found this blog. Thank you for sharing with us,I too always learn something new from your post. how to say kiolopobgofit
Sangat menyukai pendekatan Anda dalam menangani topik ini. Sangat terperinci dan informatif! DURABELT EP 125 Rubber Conveyor Belts 5 Ply BW 600
I’m happy I located this blog! From time to time, students want to cognitive the keys of productive literary essays composing. Your first-class knowledge about this good post can become a proper basis for such people. nice one money6x.com save money
Looking for a reliable roofer in San Jose? Campbell Roofing provides expert roof repair, replacement, and maintenance with high-quality materials and exceptional service. Get a free estimate today for durable, long-lasting roofing solutions!