Authors: Wissam Antoun, Fady Baly, Hazem Hajj
AraBERT is an Arabic pretrained language model based on Google’s BERT architecture. AraBERT uses the same BERT-Base config. More details are available in the AraBERT PAPER and in the AraBERT Meetup
There is two versions of the model AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the Farasa Segmenter.
The model was trained on ~70M sentences or ~23GB of Arabic text with ~3B words.
Source Code Repository: https://github.com/aub-mind/arabert
Paper: https://www.aclweb.org/anthology/2020.osact-1.2.pdf
Results (Accuracy)
We evaluate both AraBERT models on different downstream tasks and compare it to mBERT, and other state of the art models (To the extent of our knowledge). The Tasks were Sentiment Analysis on 6 different datasets (HARD, ASTD-Balanced, ArsenTD-Lev, LABR, ArSaS), Named Entity Recognition with the ANERcorp, and Arabic Question Answering on Arabic-SQuAD and ARCD
| Task | prev. SOTA | mBERT | AraBERTv0.1 | AraBERTv1 |
|---|---|---|---|---|
| HARD | 95.7 ElJundi et.al. | 95.7 | 96.2 | 96.1 |
| ASTD | 86.5 ElJundi et.al. | 80.1 | 92.2 | 92.6 |
| ArsenTD-Lev | 52.4 ElJundi et.al. | 51 | 58.9 | 59.4 |
| AJGT | 93 Dahou et.al. | 83.6 | 93.1 | 93.8 |
| LABR | 87.5 Dahou et.al. | 83 | 85.9 | 86.7 |
| ANERcorp | 81.7 (BiLSTM-CRF) | 78.4 | 84.2 | 81.9 |
| ARCD | mBERT | EM:34.2 F1: 61.3 | EM:51.14 F1:82.13 | EM:54.84 F1: 82.15 |
Model Weights and Vocab Download
| Models | AraBERTv0.1 | AraBERTv1 |
|---|---|---|
| TensorFlow | Drive Link | Drive Link |
| PyTorch | Drive_Link | Drive_Link |
You can find the PyTorch models in HuggingFace’s Transformer Library under the aubmindlab username
If you used this model please cite us as:
@inproceedings{antoun2020arabert,
title={AraBERT: Transformer-based Model for Arabic Language Understanding},
author={Antoun, Wissam and Baly, Fady and Hajj, Hazem},
booktitle={LREC 2020 Workshop Language Resources and Evaluation Conference 11--16 May 2020},
pages={9}
}
Acknowledgments
Thanks to TensorFlow Research Cloud (TFRC) for the free access to Cloud TPUs, couldn’t have done it without this program, and to the AUB MIND Lab Members for the continous support. Also thanks to Yakshof and Assafir for data and storage access. Another thanks for Habib Rahal (https://www.behance.net/rahalhabib), for putting a face to AraBERT.
Contacts
Wissam Antoun: Linkedin | Twitter | Github | wfa07@mail.aub.edu | wissam.antoun@gmail.com
Fady Baly: Linkedin | Twitter | Github | fgb06@mail.aub.edu | baly.fady@gmail.com
We are looking for sponsors to train BERT-Large and other Transformer models, the sponsor only needs to cover to data storage and compute cost of the generating the pretraining data
This information is magnificent. I understand and respect your clear-cut points. I am impressed with your writing style and how well you express your thoughts. خودکار فلزی
sangat menarik dan informatif terus update artikel terbarunya DURAJOINT Jaw Couplings
Took me time to understand all of the comments, but I seriously enjoyed the write-up. It proved being really helpful to me and Im positive to all of the commenters right here! Its constantly nice when you can not only be informed, but also entertained! I am certain you had enjoyable writing this write-up. Crypto Wallets
Saya suka cara Anda mengulas topik ini. Sangat jelas dan mudah dimengerti. Genset Genpac Open / Silent Mitsubishi 1250 kVA GMI1250
Terimakasih atas informasi artikelnya menarik sekali DURAJOINT Rubber FCL Couplings
شرایط ثبت نام بورس خارج از کشور در آزمون دکتری فرصتی استثنایی برای ادامه تحصیل در دانشگاههای برتر دنیا و ارتقای سطح علمی کشور محسوب میشود.
Made to measure curtains… […]check out the sites listed below, worth a read for interiors and rugs enthusiasts[…]… daftar slot
We are tied directly into the sate’s renewal database which allows us to process your request almost instantly. Crypto Security
pertamax gan terimakasih untuk infonya Gravity Rollers
LU88không chỉ là một nền tảng cá cược trực tuyến, mà còn là xu hướng dẫn đầu trong năm 2025. Với hệ thống giải trí đa dạng, giao diện hiện đại và chính sách ưu đãi hấp dẫn, LU88 hứa hẹn mang đến cho người chơi những trải nghiệm đỉnh cao và cơ hội chiến thắng không giới hạn.
Shop the High Quality Essentials Tracksuit At a sale price, Upto 40% Off on Online Essentials Tracksuit Store. Fast shipping Worldwild. https://essentialstracksuitus.com/tracksuit/
Shop the High Quality Essentials Tracksuit At a sale price, Upto 40% Off on Online Essentials Tracksuit Store. Fast shipping Worldwild.https://essentialstracksuitus.com/tracksuit/
Immigration… […]the time to read or visit the content or sites we have linked to below the[…]… transfer Estepona airport
VN88 là một trong những nhà cái cá cược trực tuyến hàng đầu tại Việt Nam và châu Á, thu hút hàng triệu người chơi tham gia. Không chỉ sở hữu một nền tảng giao dịch nhanh chóng và an toàn, VN88 còn cung cấp đa dạng trò chơi cá cược, từ game bài cho đến thể thao và lô đề. Với các chương trình khuyến mãi hấp dẫn cùng đội ngũ chăm sóc khách hàng chuyên nghiệp, VN88 thực sự mang đến trải nghiệm tuyệt vời cho người dùng.
Terima kasih atas informasinya. Saya belajar banyak dari artikel ini. Genset Genpac Open / Silent Mitsubishi 1000 kVA GMI1000
پاسخنامه آزمون دکتری سندی است که توسط سازمان سنجش آموزش کشور پس از برگزاری آزمون دکتری منتشر میشود و حاوی پاسخهای صحیح تمامی سوالات آزمون در تمامی رشتهها است.
آدرس دفاتر گزینش نیروی هوایی ارتشپیوستن به نیروی هوایی ارتش جمهوری اسلامی ایران، آرزوی بسیاری از جوانان با انگیزه و وطندوست است.
Took me time to understand all of the comments, but I seriously enjoyed the write-up. It proved being really helpful to me and Im positive to all of the commenters right here! Its constantly nice when you can not only be informed, but also entertained! I am certain you had enjoyable writing this write-up. icryptox
A blog like yours should be earning much money from adsense..-.,” situs togel online
The following is found 100 % most suitable. Every one of small-scale aspects are written and wide range of past recognizing. I want it plenty. https://net8877.com/
سامانه برخط پسادکتری یک بستر آنلاین است که به منظور تسهیل و شفافسازی فرآیند جذب و بکارگیری محققان پسادکتری در دانشگاهها، مراکز پژوهشی و شرکتهای دانشبنیان طراحی شده است.
I am really enjoying reading your well written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work. Lovelolablog Codes
They offer a wide range of auto glass services, including windshield repair and replacement. Contact today
Thanks much for another information. It isn?t easy that sort expertise reading through homework, i am keeping an eye out for. net88
Sangat bermanfaat! Saya akan kembali lagi untuk membaca artikel lainnya. Genset Genpac Open / Silent Mitsubishi 800 kVA GMI800
ثبت نام استخدامی پدافند هوایی ارتشنیروی پدافند هوایی ارتش جمهوری اسلامی ایران، به عنوان یکی از ارکان اصلی دفاع از حریم هوایی کشور، همواره در تلاش است تا با جذب جوانان متعهد، متخصص و با انگیزه، توان دفاعی خود را ارتقا بخشد.
Might be likewise a reasonably wonderful short article that we all most certainly most popular assessing. Might be no day-after-day in which watch the possibility to look through unit. https://s666casino.org/
Indulge in a VIP Hummer Safari Dubai for a premium desert experience.
I beloved as much as you will receive performed right here. The comic strip is attractive, your authored material stylish. nonetheless, you command get bought an nervousness over that you would like be handing over the following. unwell without a doubt come more before once more since precisely the same nearly very steadily inside of case you defend this increase. net88
Took me time to understand all of the comments, but I seriously enjoyed the write-up. It proved being really helpful to me and Im positive to all of the commenters right here! Its constantly nice when you can not only be informed, but also entertained! I am certain you had enjoyable writing this write-up. remote anydeskabramsbleepingcomputer
حداقل معدل کارشناسی ارشد برای دکتری فارغالتحصیلان دوره کارشناسی ارشد که علاقمند به ادامه تحصیل در مقطع دکتری تخصصی هستند، باید توجه داشته باشند که پذیرش در این مقطع، فرآیندی نیمه متمرکز شامل آزمون کتبی و مصاحبه است.
Immediately the website might irrefutably obtain famous one of the most associated with publishing customers, due to the persistent articles or just crucial evaluations. https://net8877.com/
I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post. 토토사이트
Currently, Indonesia is a country that is developing rapidly because many Indonesian people find it very easy to make money by playing online games on the asianbet88 site, this site is the official site belonging to the Indonesian state.
Artikel yang menarik! Saya sudah menambahkan situs ini ke bookmark saya. Genset Genpac Open / Silent Mitsubishi 750 kVA GMI750
informasi yang bagus sekali semoga info ini bermanfaat FLEXCO Belt Fasteners
سامانه مهارت یار دانشگاه آزاد پلتفرمی آنلاین است که با هدف ارتقای سطح کیفی آموزش و پرورش اساتید توانمند، توسط باشگاه پژوهشگران جوان و نخبگان این دانشگاه راهاندازی شده است.
saya kagum dengan artikel ini banyak sekali yang berminat membaca artikel ini WHALE Belt Fasteners
Informasinya sangat lengkap. Saya tidak sabar untuk membaca artikel lainnya. Genset Genpac Open / Silent Mitsubishi 650 kVA GMI650
دکتری استاد محور دانشگاه پیام نور روشی برای پذیرش دانشجویان دکتری است که در آن اساتید واجد شرایط، دانشجوی دکتری خود را بدون آزمون و بر اساس رزومه تحصیلی، سوابق پژوهشی و مصاحبه انتخاب میکنند.
saya berterimakasih karena website ini selalu memberikan hal menarik kepada kami DURALINK Roller Chains RS 80-3
Wholesale Cheap Handbags Will you be ok merely repost this on my site? I’ve to allow credit where it can be due. Have got a great day! scatter hitam
You can’t go back and change the beginning, but you can start where you are and change the ending. rr88 link
https://vebotv2.com/. VEBOTV – Experience the top online football channel and a variety of world-class sports completely free. With a commitment to leading quality, it ensures you enjoy the smoothest and sharpest matches.
How do top-followed accounts handle social media burnout?
Top-followed accounts often face social media burnout due to the intense demands of constant content creation, engagement, and maintaining a public persona. To manage burnout, many adopt the following strategies:
Scheduled Breaks: Influencers and creators often take planned breaks from social media to rest and recharge. Some disconnect for days or weeks to focus on their mental health, while others simply reduce their posting frequency.
Outsourcing Tasks: Many hire teams—such as social media managers, content creators, or virtual assistants—to handle repetitive tasks like posting, responding to messages, and tracking analytics, giving them more time for personal care and creative work.
Limiting Engagement: Setting boundaries around how often and when they engage with followers can help reduce the overwhelming feeling of being “on” all the time. Some may limit time spent on apps to avoid burnout.
Authenticity and Transparency: When dealing with burnout, influencers often openly communicate with their audience about the challenges they face. This vulnerability helps to humanize them and strengthens their relationship with followers.
Diversifying Content: Experimenting with new types of content or branching into other platforms can relieve creative pressure. It keeps their work exciting and reduces the risk of feeling stagnant.
Prioritizing Self-Care: Influencers often build self-care routines into their schedules—whether through exercise, meditation, therapy, or spending time with loved ones—to maintain their mental well-being.
By balancing their public life with personal time and self-care, top-followed accounts can handle burnout and continue to create content in a sustainable way.
htttps://topfollow.gold
https://vebotv2.com/. VEBOTV – Experience the top online football channel and a variety of world-class sports completely free. With a commitment to leading quality, it ensures you enjoy the smoothest and sharpest matches.
https://topfollow.gold
How do top-followed accounts handle social media burnout?
Top-followed accounts often face social media burnout due to the intense demands of constant content creation, engagement, and maintaining a public persona. To manage burnout, many adopt the following strategies:
Scheduled Breaks: Influencers and creators often take planned breaks from social media to rest and recharge. Some disconnect for days or weeks to focus on their mental health, while others simply reduce their posting frequency.
Outsourcing Tasks: Many hire teams—such as social media managers, content creators, or virtual assistants—to handle repetitive tasks like posting, responding to messages, and tracking analytics, giving them more time for personal care and creative work.
Limiting Engagement: Setting boundaries around how often and when they engage with followers can help reduce the overwhelming feeling of being “on” all the time. Some may limit time spent on apps to avoid burnout.
Authenticity and Transparency: When dealing with burnout, influencers often openly communicate with their audience about the challenges they face. This vulnerability helps to humanize them and strengthens their relationship with followers.
Diversifying Content: Experimenting with new types of content or branching into other platforms can relieve creative pressure. It keeps their work exciting and reduces the risk of feeling stagnant.
Prioritizing Self-Care: Influencers often build self-care routines into their schedules—whether through exercise, meditation, therapy, or spending time with loved ones—to maintain their mental well-being.
By balancing their public life with personal time and self-care, top-followed accounts can handle burnout and continue to create content in a sustainable way.
informasi ini sangat bagus sekali terimakasih telah berbagi, terus update DURALINK Roller Chains British 10B-1
Credit card cashing service is a quick and safe way to make cash using your own credit card. There is no complicated document review, so the service can be processed and deposited quickly 신용카드 현금화
مای مدیو کارنامه سایت مای مدیو با عنوان پنجره واحد خدمات الکترونیک آموزش و پرورش، در راستای تحقق دولت الکترونیک و با هدف تسهیل و تسریع امور آموزشی و پرورشی، توسط وزارت آموزش و پرورش راهاندازی شده است.